[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:31:01 dist-test-slave-6zbq krb5kdc[7351](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:31:01 dist-test-slave-6zbq krb5kdc[7351](info): set up 2 sockets
Aug 06 06:31:01 dist-test-slave-6zbq krb5kdc[7351](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260806 06:31:03.117759 7341 mini_kdc.cc:121] Time spent starting KDC: real 1.256s user 0.006s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Aug 06 06:31:03 dist-test-slave-6zbq krb5kdc[7351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997863, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:31:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:31:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.43.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
I20260806 06:31:03.240296 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:38641
--webserver_interface=127.7.43.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:42039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.43.126:38641
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:31:03.361891 7368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:03.362118 7368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:03.362154 7368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:03.363400 7368 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:31:03.363443 7368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:03.363461 7368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:03.363477 7368 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:31:03.363492 7368 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:31:03.365336 7368 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:42039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.43.126:38641
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:38641
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.43.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7368
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:03.365849 7368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:03.366160 7368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:03.373034 7377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:03.373132 7368 server_base.cc:1034] running on GCE node
W20260806 06:31:03.374383 7376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:03.374934 7380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:03.375240 7368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:03.375620 7368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:03.376765 7368 hybrid_clock.cc:648] HybridClock initialized: now 1785997863376739 us; error 70 us; skew 500 ppm
Aug 06 06:31:03 dist-test-slave-6zbq krb5kdc[7351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997863, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:03.383546 7368 init.cc:377] Logged in from keytab as kudu/127.7.43.126@KRBTEST.COM (short username kudu)
I20260806 06:31:03.384146 7368 webserver.cc:460] Webserver started at http://127.7.43.126:43463/ using document root <none> and password file <none>
I20260806 06:31:03.384403 7368 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:03.384452 7368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:03.384594 7368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:03.385910 7368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/master-0/data/instance:
uuid: "d4df65c3cb2a4fb5af4269ec0978fb78"
format_stamp: "Formatted at 2026-08-06 06:31:03 on dist-test-slave-6zbq"
server_key: "2cce69f29709b829251f7d4529441c39"
server_key_iv: "42b0feaf07e047e7284f3daa8425f6d5"
server_key_version: "encryptionkey@0"
I20260806 06:31:03.386651 7368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/master-0/wal/instance:
uuid: "d4df65c3cb2a4fb5af4269ec0978fb78"
format_stamp: "Formatted at 2026-08-06 06:31:03 on dist-test-slave-6zbq"
server_key: "2cce69f29709b829251f7d4529441c39"
server_key_iv: "42b0feaf07e047e7284f3daa8425f6d5"
server_key_version: "encryptionkey@0"
I20260806 06:31:03.388422 7368 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 06:31:03.389446 7387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:03.389710 7368 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:31:03.389791 7368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "d4df65c3cb2a4fb5af4269ec0978fb78"
format_stamp: "Formatted at 2026-08-06 06:31:03 on dist-test-slave-6zbq"
server_key: "2cce69f29709b829251f7d4529441c39"
server_key_iv: "42b0feaf07e047e7284f3daa8425f6d5"
server_key_version: "encryptionkey@0"
I20260806 06:31:03.389846 7368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:03.403390 7368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:03.405341 7368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:03.405479 7368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:03.415192 7368 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.126:38641
I20260806 06:31:03.415333 7458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.126:38641 every 8 connection(s)
I20260806 06:31:03.415839 7368 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:31:03.416635 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 7368
I20260806 06:31:03.416764 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/master-0/wal/instance
I20260806 06:31:03.417603 7341 external_mini_cluster.cc:1428] Setting key 06e443d8bd2392030f35576f036e3613
I20260806 06:31:03.417855 7459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:03.422734 7459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4df65c3cb2a4fb5af4269ec0978fb78: Bootstrap starting.
I20260806 06:31:03.424365 7459 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d4df65c3cb2a4fb5af4269ec0978fb78: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:03.425238 7459 log.cc:824] T 00000000000000000000000000000000 P d4df65c3cb2a4fb5af4269ec0978fb78: Log is configured to *not* fsync() on all Append() calls
I20260806 06:31:03.426216 7459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4df65c3cb2a4fb5af4269ec0978fb78: No bootstrap required, opened a new log
I20260806 06:31:03.430974 7459 raft_consensus.cc:348] T 00000000000000000000000000000000 P d4df65c3cb2a4fb5af4269ec0978fb78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4df65c3cb2a4fb5af4269ec0978fb78" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 38641 } }
I20260806 06:31:03.431180 7459 raft_consensus.cc:374] T 00000000000000000000000000000000 P d4df65c3cb2a4fb5af4269ec0978fb78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:03.431269 7459 raft_consensus.cc:729] T 00000000000000000000000000000000 P d4df65c3cb2a4fb5af4269ec0978fb78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4df65c3cb2a4fb5af4269ec0978fb78, State: Initialized, Role: FOLLOWER
I20260806 06:31:03.431432 7459 consensus_queue.cc:260] T 00000000000000000000000000000000 P d4df65c3cb2a4fb5af4269ec0978fb78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4df65c3cb2a4fb5af4269ec0978fb78" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 38641 } }
I20260806 06:31:03.431571 7459 raft_consensus.cc:388] T 00000000000000000000000000000000 P d4df65c3cb2a4fb5af4269ec0978fb78 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:31:03.431600 7459 raft_consensus.cc:482] T 00000000000000000000000000000000 P d4df65c3cb2a4fb5af4269ec0978fb78 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:31:03.431649 7459 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d4df65c3cb2a4fb5af4269ec0978fb78 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:03.432418 7459 raft_consensus.cc:504] T 00000000000000000000000000000000 P d4df65c3cb2a4fb5af4269ec0978fb78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4df65c3cb2a4fb5af4269ec0978fb78" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 38641 } }
I20260806 06:31:03.432641 7459 leader_election.cc:302] T 00000000000000000000000000000000 P d4df65c3cb2a4fb5af4269ec0978fb78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d4df65c3cb2a4fb5af4269ec0978fb78; no voters:
I20260806 06:31:03.432886 7459 leader_election.cc:288] T 00000000000000000000000000000000 P d4df65c3cb2a4fb5af4269ec0978fb78 [CANDIDATE]: Term 1 election: Requested vote from peers
Aug 06 06:31:03 dist-test-slave-6zbq krb5kdc[7351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997863, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:31:03.432960 7467 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d4df65c3cb2a4fb5af4269ec0978fb78 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:31:03.433249 7467 raft_consensus.cc:686] T 00000000000000000000000000000000 P d4df65c3cb2a4fb5af4269ec0978fb78 [term 1 LEADER]: Becoming Leader. State: Replica: d4df65c3cb2a4fb5af4269ec0978fb78, State: Running, Role: LEADER
I20260806 06:31:03.433383 7467 consensus_queue.cc:237] T 00000000000000000000000000000000 P d4df65c3cb2a4fb5af4269ec0978fb78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4df65c3cb2a4fb5af4269ec0978fb78" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 38641 } }
I20260806 06:31:03.433391 7459 sys_catalog.cc:564] T 00000000000000000000000000000000 P d4df65c3cb2a4fb5af4269ec0978fb78 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:31:03.434252 7468 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4df65c3cb2a4fb5af4269ec0978fb78 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d4df65c3cb2a4fb5af4269ec0978fb78. Latest consensus state: current_term: 1 leader_uuid: "d4df65c3cb2a4fb5af4269ec0978fb78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4df65c3cb2a4fb5af4269ec0978fb78" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 38641 } } }
I20260806 06:31:03.434365 7468 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4df65c3cb2a4fb5af4269ec0978fb78 [sys.catalog]: This master's current role is: LEADER
I20260806 06:31:03.434289 7471 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4df65c3cb2a4fb5af4269ec0978fb78 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d4df65c3cb2a4fb5af4269ec0978fb78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4df65c3cb2a4fb5af4269ec0978fb78" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 38641 } } }
I20260806 06:31:03.434538 7471 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4df65c3cb2a4fb5af4269ec0978fb78 [sys.catalog]: This master's current role is: LEADER
I20260806 06:31:03.434960 7476 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:31:03.436472 7476 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:31:03.439057 7476 catalog_manager.cc:1347] Generated new cluster ID: ba4fb59b776b4bdfa7d2e6f88b9d7cbd
I20260806 06:31:03.439110 7476 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:31:03.439422 7462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:03.418992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41288 (local address 127.7.43.126:38641)
0806 06:31:03.419401 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:03.419412 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:03.419751 (+ 339us) server_negotiation.cc:407] Connection header received
0806 06:31:03.423355 (+ 3604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:03.423364 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:03.423487 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:03.423656 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:03.425633 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:03.427519 (+ 1886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:03.428737 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:03.429450 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:03.429613 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:03.433774 (+ 4161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:03.433779 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:03.433780 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:03.433791 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:03.436932 (+ 3141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:03.437413 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:03.437415 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:03.437415 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:03.437490 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:03.437779 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:03.437780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:03.437780 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:03.438076 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:03.438250 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:03.438944 (+ 694us) server_negotiation.cc:299] Negotiation successful
0806 06:31:03.439177 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":160,"threads_started":1}
I20260806 06:31:03.455693 7476 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:31:03.456393 7476 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:31:03.465572 7476 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d4df65c3cb2a4fb5af4269ec0978fb78: Generated new TSK 0
I20260806 06:31:03.465751 7476 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.43.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab.
Entry for principal kudu/127.7.43.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab.
I20260806 06:31:03.491987 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.65:0
--local_ip_for_outbound_sockets=127.7.43.65
--webserver_interface=127.7.43.65
--webserver_port=0
--tserver_master_addrs=127.7.43.126:38641
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab
--principal=kudu/127.7.43.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:42039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:31:03.605371 7502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:03.605547 7502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:03.605571 7502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:03.606734 7502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:03.606777 7502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:03.606803 7502 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.65
I20260806 06:31:03.608194 7502 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:42039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab
--principal=kudu/127.7.43.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.43.65
--webserver_port=0
--tserver_master_addrs=127.7.43.126:38641
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7502
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.65
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:03.608461 7502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:03.608685 7502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:03.613574 7509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:03.613528 7508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:03.613838 7502 server_base.cc:1034] running on GCE node
W20260806 06:31:03.613883 7512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:03.614104 7502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:03.614324 7502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:03.615505 7502 hybrid_clock.cc:648] HybridClock initialized: now 1785997863615488 us; error 32 us; skew 500 ppm
Aug 06 06:31:03 dist-test-slave-6zbq krb5kdc[7351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997863, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:03.617865 7502 init.cc:377] Logged in from keytab as kudu/127.7.43.65@KRBTEST.COM (short username kudu)
I20260806 06:31:03.618294 7502 webserver.cc:460] Webserver started at http://127.7.43.65:36723/ using document root <none> and password file <none>
I20260806 06:31:03.618517 7502 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:03.618561 7502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:03.618665 7502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:03.619661 7502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/ts-0/data/instance:
uuid: "38bc6140ba1a476089b05a59f40a1a40"
format_stamp: "Formatted at 2026-08-06 06:31:03 on dist-test-slave-6zbq"
server_key: "4ad124052f30404c14b8b4575174c0ea"
server_key_iv: "c60ef50d90745ccac028b849149d5f73"
server_key_version: "encryptionkey@0"
I20260806 06:31:03.619978 7502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/ts-0/wal/instance:
uuid: "38bc6140ba1a476089b05a59f40a1a40"
format_stamp: "Formatted at 2026-08-06 06:31:03 on dist-test-slave-6zbq"
server_key: "4ad124052f30404c14b8b4575174c0ea"
server_key_iv: "c60ef50d90745ccac028b849149d5f73"
server_key_version: "encryptionkey@0"
I20260806 06:31:03.621188 7502 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 06:31:03.621927 7519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:03.622107 7502 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 06:31:03.622189 7502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/ts-0/wal
uuid: "38bc6140ba1a476089b05a59f40a1a40"
format_stamp: "Formatted at 2026-08-06 06:31:03 on dist-test-slave-6zbq"
server_key: "4ad124052f30404c14b8b4575174c0ea"
server_key_iv: "c60ef50d90745ccac028b849149d5f73"
server_key_version: "encryptionkey@0"
I20260806 06:31:03.622262 7502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:03.646489 7502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:03.648190 7502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:03.648329 7502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:03.648602 7502 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:31:03.648979 7502 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:31:03.649065 7502 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:03.649101 7502 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:31:03.649125 7502 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:03.657987 7502 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.65:34217
I20260806 06:31:03.658038 7687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.65:34217 every 8 connection(s)
I20260806 06:31:03.658334 7502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
Aug 06 06:31:03 dist-test-slave-6zbq krb5kdc[7351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997863, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.65@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:31:03.667104 7462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:03.659221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:48829 (local address 127.7.43.126:38641)
0806 06:31:03.659283 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:03.659285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:03.659449 (+ 164us) server_negotiation.cc:407] Connection header received
0806 06:31:03.659955 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:03.659957 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:03.659969 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:03.660016 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:03.661132 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:03.661635 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:03.662623 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:03.662825 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:03.662881 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:03.664832 (+ 1951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:03.664833 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:03.664834 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:03.664841 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:03.665896 (+ 1055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:03.666245 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:03.666246 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:03.666246 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:03.666298 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:03.666523 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:03.666524 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:03.666524 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:03.666627 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:03.666682 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:03.666930 (+ 248us) server_negotiation.cc:299] Negotiation successful
0806 06:31:03.667017 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 06:31:03.667243 7690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:03.659182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38641 (local address 127.7.43.65:48829)
0806 06:31:03.659327 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:03.659370 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:03.659856 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:03.660098 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:03.660102 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:03.660834 (+ 732us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:03.661021 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:03.661026 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:03.661730 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:03.661732 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:03.662534 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:03.662537 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:03.662894 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:03.662895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:03.663015 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:03.663456 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:03.663464 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:03.664742 (+ 1278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:03.665965 (+ 1223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:03.665972 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:03.665974 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:03.666174 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:03.666368 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:03.666369 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:03.666369 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:03.666467 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:03.666679 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:03.666686 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:03.666816 (+ 130us) client_negotiation.cc:769] Sending connection context
0806 06:31:03.666901 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 06:31:03.667046 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":69,"threads_started":1}
I20260806 06:31:03.667507 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 7502
I20260806 06:31:03.667582 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/ts-0/wal/instance
I20260806 06:31:03.667758 7341 external_mini_cluster.cc:1428] Setting key 60fb0e2f051a6a663e929e7d7b5eeac0
I20260806 06:31:03.667774 7688 heartbeater.cc:347] Connected to a master server at 127.7.43.126:38641
I20260806 06:31:03.667863 7688 heartbeater.cc:464] Registering TS with master...
I20260806 06:31:03.668033 7688 heartbeater.cc:511] Master 127.7.43.126:38641 requested a full tablet report, sending...
I20260806 06:31:03.668924 7414 ts_manager.cc:194] Registered new tserver with Master: 38bc6140ba1a476089b05a59f40a1a40 (127.7.43.65:34217)
I20260806 06:31:03.669939 7414 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.65@KRBTEST.COM'} at 127.7.43.65:48829
WARNING: no policy specified for kudu/127.7.43.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab.
Entry for principal kudu/127.7.43.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab.
I20260806 06:31:03.692209 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.66:0
--local_ip_for_outbound_sockets=127.7.43.66
--webserver_interface=127.7.43.66
--webserver_port=0
--tserver_master_addrs=127.7.43.126:38641
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab
--principal=kudu/127.7.43.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:42039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:31:03.771445 7697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:03.771620 7697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:03.771649 7697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:03.772727 7697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:03.772765 7697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:03.772801 7697 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.66
I20260806 06:31:03.774592 7697 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:42039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab
--principal=kudu/127.7.43.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.43.66
--webserver_port=0
--tserver_master_addrs=127.7.43.126:38641
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7697
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.66
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:03.774816 7697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:03.775002 7697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:03.779619 7709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:03.779640 7705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:03.779640 7704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:03.779733 7697 server_base.cc:1034] running on GCE node
I20260806 06:31:03.779898 7697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:03.780117 7697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:03.781241 7697 hybrid_clock.cc:648] HybridClock initialized: now 1785997863781229 us; error 30 us; skew 500 ppm
Aug 06 06:31:03 dist-test-slave-6zbq krb5kdc[7351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997863, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:03.783468 7697 init.cc:377] Logged in from keytab as kudu/127.7.43.66@KRBTEST.COM (short username kudu)
I20260806 06:31:03.783859 7697 webserver.cc:460] Webserver started at http://127.7.43.66:44557/ using document root <none> and password file <none>
I20260806 06:31:03.784034 7697 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:03.784070 7697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:03.784153 7697 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:03.785036 7697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/ts-1/data/instance:
uuid: "fd109aa1590e4e74a67b09acf50bff41"
format_stamp: "Formatted at 2026-08-06 06:31:03 on dist-test-slave-6zbq"
server_key: "edf752ea63dd2bdd1b8e7603f4aa1a65"
server_key_iv: "22d69e6953c25365b1e31dfd6d595f6f"
server_key_version: "encryptionkey@0"
I20260806 06:31:03.785324 7697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/ts-1/wal/instance:
uuid: "fd109aa1590e4e74a67b09acf50bff41"
format_stamp: "Formatted at 2026-08-06 06:31:03 on dist-test-slave-6zbq"
server_key: "edf752ea63dd2bdd1b8e7603f4aa1a65"
server_key_iv: "22d69e6953c25365b1e31dfd6d595f6f"
server_key_version: "encryptionkey@0"
I20260806 06:31:03.786440 7697 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:31:03.787062 7718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:03.787200 7697 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:31:03.787274 7697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/ts-1/wal
uuid: "fd109aa1590e4e74a67b09acf50bff41"
format_stamp: "Formatted at 2026-08-06 06:31:03 on dist-test-slave-6zbq"
server_key: "edf752ea63dd2bdd1b8e7603f4aa1a65"
server_key_iv: "22d69e6953c25365b1e31dfd6d595f6f"
server_key_version: "encryptionkey@0"
I20260806 06:31:03.787326 7697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:03.806524 7697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:03.808148 7697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:03.808251 7697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:03.808413 7697 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:31:03.808681 7697 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:31:03.808714 7697 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:03.808734 7697 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:31:03.808758 7697 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:03.817102 7697 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.66:39663
I20260806 06:31:03.817142 7876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.66:39663 every 8 connection(s)
I20260806 06:31:03.817410 7697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
Aug 06 06:31:03 dist-test-slave-6zbq krb5kdc[7351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997863, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.66@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:31:03.825546 7462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:03.818150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:37469 (local address 127.7.43.126:38641)
0806 06:31:03.818200 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:03.818202 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:03.818369 (+ 167us) server_negotiation.cc:407] Connection header received
0806 06:31:03.818850 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:03.818852 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:03.818865 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:03.818921 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:03.819853 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:03.820337 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:03.821272 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:03.821459 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:03.821512 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:03.823395 (+ 1883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:03.823396 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:03.823396 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:03.823403 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:03.824411 (+ 1008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:03.824796 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:03.824797 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:03.824797 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:03.824891 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:03.825063 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:03.825063 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:03.825064 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:03.825192 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:03.825237 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:03.825425 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 06:31:03.825485 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 06:31:03.825687 7880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:03.818098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38641 (local address 127.7.43.66:37469)
0806 06:31:03.818240 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:03.818282 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:03.818754 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:03.818990 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:03.818994 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:03.819574 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:03.819757 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:03.819762 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:03.820417 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:03.820418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:03.821189 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:03.821192 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:03.821524 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:03.821525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:03.821639 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:03.822002 (+ 363us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:03.822010 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:03.823307 (+ 1297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:03.824475 (+ 1168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:03.824480 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:03.824482 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:03.824728 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:03.824938 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:03.824938 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:03.824939 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:03.825025 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:03.825233 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:03.825235 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:03.825338 (+ 103us) client_negotiation.cc:769] Sending connection context
0806 06:31:03.825410 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 06:31:03.825519 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":69,"threads_started":1}
I20260806 06:31:03.826103 7878 heartbeater.cc:347] Connected to a master server at 127.7.43.126:38641
I20260806 06:31:03.826179 7878 heartbeater.cc:464] Registering TS with master...
I20260806 06:31:03.826311 7878 heartbeater.cc:511] Master 127.7.43.126:38641 requested a full tablet report, sending...
I20260806 06:31:03.826645 7414 ts_manager.cc:194] Registered new tserver with Master: fd109aa1590e4e74a67b09acf50bff41 (127.7.43.66:39663)
I20260806 06:31:03.826972 7414 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.66@KRBTEST.COM'} at 127.7.43.66:37469
I20260806 06:31:03.827445 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 7697
I20260806 06:31:03.827523 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/ts-1/wal/instance
I20260806 06:31:03.827670 7341 external_mini_cluster.cc:1428] Setting key c7dd78c049f701f731a45c29de80304f
WARNING: no policy specified for kudu/127.7.43.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab.
Entry for principal kudu/127.7.43.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab.
I20260806 06:31:03.850559 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.67:0
--local_ip_for_outbound_sockets=127.7.43.67
--webserver_interface=127.7.43.67
--webserver_port=0
--tserver_master_addrs=127.7.43.126:38641
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab
--principal=kudu/127.7.43.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:42039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:31:03.972558 7883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:03.972754 7883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:03.972785 7883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:03.973930 7883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:03.973970 7883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:03.974005 7883 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.67
I20260806 06:31:03.975497 7883 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:42039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab
--principal=kudu/127.7.43.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.43.67
--webserver_port=0
--tserver_master_addrs=127.7.43.126:38641
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7883
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.67
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:03.975718 7883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:03.975919 7883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:03.980536 7894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:03.980545 7890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:03.980677 7883 server_base.cc:1034] running on GCE node
W20260806 06:31:03.980631 7891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:03.980921 7883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:03.981197 7883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:03.982331 7883 hybrid_clock.cc:648] HybridClock initialized: now 1785997863982319 us; error 31 us; skew 500 ppm
Aug 06 06:31:03 dist-test-slave-6zbq krb5kdc[7351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997863, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:03.984592 7883 init.cc:377] Logged in from keytab as kudu/127.7.43.67@KRBTEST.COM (short username kudu)
I20260806 06:31:03.985036 7883 webserver.cc:460] Webserver started at http://127.7.43.67:34093/ using document root <none> and password file <none>
I20260806 06:31:03.985232 7883 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:03.985278 7883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:03.985399 7883 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:03.986476 7883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/ts-2/data/instance:
uuid: "8e3427ddebb14df398ea7688c1ab775d"
format_stamp: "Formatted at 2026-08-06 06:31:03 on dist-test-slave-6zbq"
server_key: "10671f5bb7331aab57c1c485e8c10ae0"
server_key_iv: "81deaafa93cc10d895a2c2b29f16127b"
server_key_version: "encryptionkey@0"
I20260806 06:31:03.986845 7883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/ts-2/wal/instance:
uuid: "8e3427ddebb14df398ea7688c1ab775d"
format_stamp: "Formatted at 2026-08-06 06:31:03 on dist-test-slave-6zbq"
server_key: "10671f5bb7331aab57c1c485e8c10ae0"
server_key_iv: "81deaafa93cc10d895a2c2b29f16127b"
server_key_version: "encryptionkey@0"
I20260806 06:31:03.988222 7883 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260806 06:31:03.988903 7904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:03.989046 7883 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:03.989130 7883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/ts-2/wal
uuid: "8e3427ddebb14df398ea7688c1ab775d"
format_stamp: "Formatted at 2026-08-06 06:31:03 on dist-test-slave-6zbq"
server_key: "10671f5bb7331aab57c1c485e8c10ae0"
server_key_iv: "81deaafa93cc10d895a2c2b29f16127b"
server_key_version: "encryptionkey@0"
I20260806 06:31:03.989188 7883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:04.004266 7883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:04.005868 7883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:04.005990 7883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:04.006263 7883 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:31:04.006652 7883 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:31:04.006690 7883 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:04.006716 7883 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:31:04.006732 7883 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:04.019357 7883 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.67:41997
I20260806 06:31:04.019398 8061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.67:41997 every 8 connection(s)
I20260806 06:31:04.019671 7883 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
Aug 06 06:31:04 dist-test-slave-6zbq krb5kdc[7351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997863, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.67@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:31:04.025374 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 7883
I20260806 06:31:04.025453 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785997861828271-7341-0/minicluster-data/ts-2/wal/instance
I20260806 06:31:04.025600 7341 external_mini_cluster.cc:1428] Setting key 3a4d35719d1930817debeeafc2eb20ca
I20260806 06:31:04.028168 7462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:04.020335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:50671 (local address 127.7.43.126:38641)
0806 06:31:04.020374 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:04.020376 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:04.020668 (+ 292us) server_negotiation.cc:407] Connection header received
0806 06:31:04.021147 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:04.021147 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:04.021157 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:04.021204 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:04.022012 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:04.022537 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:04.023433 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:04.023605 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:04.023644 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:04.025696 (+ 2052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:04.025697 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:04.025698 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:04.025705 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:04.027192 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:04.027526 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:04.027526 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:04.027526 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:04.027571 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:04.027726 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:04.027726 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:04.027726 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:04.027820 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:04.027862 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:04.028049 (+ 187us) server_negotiation.cc:299] Negotiation successful
0806 06:31:04.028106 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 06:31:04.028326 8066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:04.020325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38641 (local address 127.7.43.67:50671)
0806 06:31:04.020465 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:04.020497 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:04.021052 (+ 555us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:04.021256 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:04.021259 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:04.021776 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:04.021950 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:04.021955 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:04.022594 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:04.022595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:04.023369 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:04.023372 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:04.023653 (+ 281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:04.023654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:04.023742 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:04.024182 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:04.024190 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:04.025603 (+ 1413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:04.027266 (+ 1663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:04.027272 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:04.027274 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:04.027470 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:04.027612 (+ 142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:04.027613 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:04.027613 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:04.027693 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:04.027859 (+ 166us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:04.027862 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:04.027960 (+ 98us) client_negotiation.cc:769] Sending connection context
0806 06:31:04.028032 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 06:31:04.028138 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":80,"threads_started":1}
I20260806 06:31:04.028872 8063 heartbeater.cc:347] Connected to a master server at 127.7.43.126:38641
I20260806 06:31:04.028944 8063 heartbeater.cc:464] Registering TS with master...
I20260806 06:31:04.029098 8063 heartbeater.cc:511] Master 127.7.43.126:38641 requested a full tablet report, sending...
I20260806 06:31:04.029428 7414 ts_manager.cc:194] Registered new tserver with Master: 8e3427ddebb14df398ea7688c1ab775d (127.7.43.67:41997)
I20260806 06:31:04.029752 7414 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.67@KRBTEST.COM'} at 127.7.43.67:50671
I20260806 06:31:04.037593 7341 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 06:31:04 dist-test-slave-6zbq krb5kdc[7351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997864, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 06:31:04 dist-test-slave-6zbq krb5kdc[7351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997864, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.43.65@KRBTEST.COM
I20260806 06:31:04.063057 8071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:04.056061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52894 (local address 127.7.43.65:34217)
0806 06:31:04.056317 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:04.056322 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:04.056331 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:31:04.056356 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:04.056358 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:04.056370 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:04.056430 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:04.056910 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:04.057584 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:04.058300 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:04.058629 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:04.058678 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:04.060708 (+ 2030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:04.060716 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:04.060716 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:04.060724 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:04.062141 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:04.062452 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:04.062454 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:04.062455 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:04.062517 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:04.062642 (+ 125us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:04.062642 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:04.062643 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:04.062752 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:04.062854 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:04.062903 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 06:31:04.062960 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":189,"threads_started":1}
W20260806 06:31:04.063498 7561 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52894
Aug 06 06:31:04 dist-test-slave-6zbq krb5kdc[7351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 06:31:04 dist-test-slave-6zbq krb5kdc[7351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.43.65@KRBTEST.COM
I20260806 06:31:04.085467 8071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:04.079637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52908 (local address 127.7.43.65:34217)
0806 06:31:04.079687 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:04.079689 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:04.079751 (+ 62us) server_negotiation.cc:407] Connection header received
0806 06:31:04.079796 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:04.079797 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:04.079806 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:04.079849 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:04.080310 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:04.080755 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:04.081454 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:04.081641 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:04.081691 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:04.083684 (+ 1993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:04.083685 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:04.083686 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:04.083693 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:04.084718 (+ 1025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:04.084982 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:04.084983 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:04.084983 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:04.085023 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:04.085147 (+ 124us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:04.085147 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:04.085148 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:04.085222 (+ 74us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:04.085298 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:04.085352 (+ 54us) server_negotiation.cc:299] Negotiation successful
0806 06:31:04.085408 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 06:31:04.085884 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 7502
I20260806 06:31:04.096575 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 7697
I20260806 06:31:04.103535 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 7883
I20260806 06:31:04.110239 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 7368
2026-08-06T06:31:04Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2277 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:31:04 dist-test-slave-6zbq krb5kdc[8080](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:31:04 dist-test-slave-6zbq krb5kdc[8080](info): set up 2 sockets
Aug 06 06:31:04 dist-test-slave-6zbq krb5kdc[8080](info): commencing operation
krb5kdc: starting...
W20260806 06:31:05.212441 7341 mini_kdc.cc:121] Time spent starting KDC: real 1.081s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Aug 06 06:31:05 dist-test-slave-6zbq krb5kdc[8080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:31:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:31:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.43.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
I20260806 06:31:05.316501 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:43039
--webserver_interface=127.7.43.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:32987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.43.126:43039
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:31:05.393894 8097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:05.394076 8097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:05.394106 8097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:05.395211 8097 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:31:05.395252 8097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:05.395279 8097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:05.395295 8097 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:31:05.395311 8097 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:31:05.396739 8097 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:32987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.43.126:43039
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:43039
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.43.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8097
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:05.396967 8097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:05.397154 8097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:05.401572 8108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:05.401700 8097 server_base.cc:1034] running on GCE node
W20260806 06:31:05.401623 8104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:05.401608 8105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:05.402025 8097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:05.402282 8097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:05.403452 8097 hybrid_clock.cc:648] HybridClock initialized: now 1785997865403436 us; error 31 us; skew 500 ppm
Aug 06 06:31:05 dist-test-slave-6zbq krb5kdc[8080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997865, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:05.405742 8097 init.cc:377] Logged in from keytab as kudu/127.7.43.126@KRBTEST.COM (short username kudu)
I20260806 06:31:05.406165 8097 webserver.cc:460] Webserver started at http://127.7.43.126:45755/ using document root <none> and password file <none>
I20260806 06:31:05.406401 8097 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:05.406450 8097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:05.406569 8097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:05.407500 8097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/master-0/data/instance:
uuid: "6cd112f4f1dc4bcc99c02418049d94e2"
format_stamp: "Formatted at 2026-08-06 06:31:05 on dist-test-slave-6zbq"
server_key: "8f7af28af4f693c77b1817500e4e1901"
server_key_iv: "b800711b99b73df3e049a468476150c6"
server_key_version: "encryptionkey@0"
I20260806 06:31:05.407819 8097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/master-0/wal/instance:
uuid: "6cd112f4f1dc4bcc99c02418049d94e2"
format_stamp: "Formatted at 2026-08-06 06:31:05 on dist-test-slave-6zbq"
server_key: "8f7af28af4f693c77b1817500e4e1901"
server_key_iv: "b800711b99b73df3e049a468476150c6"
server_key_version: "encryptionkey@0"
I20260806 06:31:05.408957 8097 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:31:05.409536 8117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:05.409680 8097 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:31:05.409770 8097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "6cd112f4f1dc4bcc99c02418049d94e2"
format_stamp: "Formatted at 2026-08-06 06:31:05 on dist-test-slave-6zbq"
server_key: "8f7af28af4f693c77b1817500e4e1901"
server_key_iv: "b800711b99b73df3e049a468476150c6"
server_key_version: "encryptionkey@0"
I20260806 06:31:05.409826 8097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:05.426574 8097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:05.428400 8097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:05.428519 8097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:05.432170 8097 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.126:43039
I20260806 06:31:05.432201 8187 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.126:43039 every 8 connection(s)
I20260806 06:31:05.432487 8097 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:31:05.433246 8189 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:05.435211 8189 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6cd112f4f1dc4bcc99c02418049d94e2: Bootstrap starting.
I20260806 06:31:05.435693 8189 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6cd112f4f1dc4bcc99c02418049d94e2: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:05.435928 8189 log.cc:824] T 00000000000000000000000000000000 P 6cd112f4f1dc4bcc99c02418049d94e2: Log is configured to *not* fsync() on all Append() calls
I20260806 06:31:05.436474 8189 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6cd112f4f1dc4bcc99c02418049d94e2: No bootstrap required, opened a new log
I20260806 06:31:05.437937 8189 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6cd112f4f1dc4bcc99c02418049d94e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cd112f4f1dc4bcc99c02418049d94e2" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 43039 } }
I20260806 06:31:05.438059 8189 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6cd112f4f1dc4bcc99c02418049d94e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:05.438086 8189 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6cd112f4f1dc4bcc99c02418049d94e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cd112f4f1dc4bcc99c02418049d94e2, State: Initialized, Role: FOLLOWER
I20260806 06:31:05.438189 8189 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6cd112f4f1dc4bcc99c02418049d94e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cd112f4f1dc4bcc99c02418049d94e2" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 43039 } }
I20260806 06:31:05.438261 8189 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6cd112f4f1dc4bcc99c02418049d94e2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:31:05.438288 8189 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6cd112f4f1dc4bcc99c02418049d94e2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:31:05.438318 8189 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6cd112f4f1dc4bcc99c02418049d94e2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:05.438792 8189 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6cd112f4f1dc4bcc99c02418049d94e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cd112f4f1dc4bcc99c02418049d94e2" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 43039 } }
I20260806 06:31:05.438876 8189 leader_election.cc:302] T 00000000000000000000000000000000 P 6cd112f4f1dc4bcc99c02418049d94e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6cd112f4f1dc4bcc99c02418049d94e2; no voters:
I20260806 06:31:05.438997 8189 leader_election.cc:288] T 00000000000000000000000000000000 P 6cd112f4f1dc4bcc99c02418049d94e2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:31:05.439049 8193 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6cd112f4f1dc4bcc99c02418049d94e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:31:05.439244 8189 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6cd112f4f1dc4bcc99c02418049d94e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:31:05.439543 8193 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6cd112f4f1dc4bcc99c02418049d94e2 [term 1 LEADER]: Becoming Leader. State: Replica: 6cd112f4f1dc4bcc99c02418049d94e2, State: Running, Role: LEADER
I20260806 06:31:05.439636 8193 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6cd112f4f1dc4bcc99c02418049d94e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cd112f4f1dc4bcc99c02418049d94e2" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 43039 } }
I20260806 06:31:05.439988 8194 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cd112f4f1dc4bcc99c02418049d94e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6cd112f4f1dc4bcc99c02418049d94e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cd112f4f1dc4bcc99c02418049d94e2" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 43039 } } }
I20260806 06:31:05.439993 8197 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cd112f4f1dc4bcc99c02418049d94e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6cd112f4f1dc4bcc99c02418049d94e2. Latest consensus state: current_term: 1 leader_uuid: "6cd112f4f1dc4bcc99c02418049d94e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cd112f4f1dc4bcc99c02418049d94e2" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 43039 } } }
I20260806 06:31:05.440147 8194 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cd112f4f1dc4bcc99c02418049d94e2 [sys.catalog]: This master's current role is: LEADER
I20260806 06:31:05.440155 8197 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cd112f4f1dc4bcc99c02418049d94e2 [sys.catalog]: This master's current role is: LEADER
I20260806 06:31:05.440747 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 8097
I20260806 06:31:05.440845 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/master-0/wal/instance
I20260806 06:31:05.441000 7341 external_mini_cluster.cc:1428] Setting key a550d8a0dedcb9ed51323d7a2464332b
W20260806 06:31:05.440955 8213 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6cd112f4f1dc4bcc99c02418049d94e2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:31:05.441581 8213 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:31:05.441720 8209 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:31:05.441892 8209 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:31:05.443233 8209 catalog_manager.cc:1347] Generated new cluster ID: 5f27b5662a464d2d97cd014763cb33cb
I20260806 06:31:05.443286 8209 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:31:05 dist-test-slave-6zbq krb5kdc[8080](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:31:05.451506 8209 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:31:05.452142 8209 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:31:05.452201 8218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:05.441525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42720 (local address 127.7.43.126:43039)
0806 06:31:05.441752 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:05.441760 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:05.441794 (+ 34us) server_negotiation.cc:407] Connection header received
0806 06:31:05.441878 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:05.441883 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:05.441924 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:05.442031 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:05.442668 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:05.444210 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:05.445290 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:05.445858 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:05.445956 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:05.448597 (+ 2641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:05.448601 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:05.448602 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:05.448615 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:05.450476 (+ 1861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:05.450838 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:05.450841 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:05.450841 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:05.450947 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:05.451145 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:05.451145 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:05.451146 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:05.451548 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:05.451736 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:05.451857 (+ 121us) server_negotiation.cc:299] Negotiation successful
0806 06:31:05.451989 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":70,"threads_started":1}
I20260806 06:31:05.454280 8209 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6cd112f4f1dc4bcc99c02418049d94e2: Generated new TSK 0
I20260806 06:31:05.454442 8209 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.43.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab.
Entry for principal kudu/127.7.43.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab.
I20260806 06:31:05.482550 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.65:0
--local_ip_for_outbound_sockets=127.7.43.65
--webserver_interface=127.7.43.65
--webserver_port=0
--tserver_master_addrs=127.7.43.126:43039
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab
--principal=kudu/127.7.43.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:32987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:31:05.561221 8222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:05.561404 8222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:05.561434 8222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:05.562556 8222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:05.562595 8222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:05.562631 8222 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.65
I20260806 06:31:05.564052 8222 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:32987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab
--principal=kudu/127.7.43.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.43.65
--webserver_port=0
--tserver_master_addrs=127.7.43.126:43039
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.65
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:05.564276 8222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:05.564462 8222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:05.568984 8228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:05.568970 8231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:05.568964 8229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:05.569181 8222 server_base.cc:1034] running on GCE node
I20260806 06:31:05.569375 8222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:05.569577 8222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:05.570705 8222 hybrid_clock.cc:648] HybridClock initialized: now 1785997865570694 us; error 24 us; skew 500 ppm
Aug 06 06:31:05 dist-test-slave-6zbq krb5kdc[8080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997865, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:05.573130 8222 init.cc:377] Logged in from keytab as kudu/127.7.43.65@KRBTEST.COM (short username kudu)
I20260806 06:31:05.573475 8222 webserver.cc:460] Webserver started at http://127.7.43.65:44275/ using document root <none> and password file <none>
I20260806 06:31:05.573642 8222 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:05.573678 8222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:05.573772 8222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:05.574712 8222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/ts-0/data/instance:
uuid: "dcea9c51487145c1bc83fec5f9e9d794"
format_stamp: "Formatted at 2026-08-06 06:31:05 on dist-test-slave-6zbq"
server_key: "01536b0e4656859f1fc1acce69f0ee7a"
server_key_iv: "6ef5296cbe9bf59e3f67bc0324b11046"
server_key_version: "encryptionkey@0"
I20260806 06:31:05.574999 8222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/ts-0/wal/instance:
uuid: "dcea9c51487145c1bc83fec5f9e9d794"
format_stamp: "Formatted at 2026-08-06 06:31:05 on dist-test-slave-6zbq"
server_key: "01536b0e4656859f1fc1acce69f0ee7a"
server_key_iv: "6ef5296cbe9bf59e3f67bc0324b11046"
server_key_version: "encryptionkey@0"
I20260806 06:31:05.576108 8222 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:31:05.576728 8240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:05.576876 8222 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:31:05.576956 8222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/ts-0/wal
uuid: "dcea9c51487145c1bc83fec5f9e9d794"
format_stamp: "Formatted at 2026-08-06 06:31:05 on dist-test-slave-6zbq"
server_key: "01536b0e4656859f1fc1acce69f0ee7a"
server_key_iv: "6ef5296cbe9bf59e3f67bc0324b11046"
server_key_version: "encryptionkey@0"
I20260806 06:31:05.577023 8222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:05.599213 8222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:05.601150 8222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:05.601289 8222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:05.601476 8222 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:31:05.601751 8222 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:31:05.601794 8222 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:05.601820 8222 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:31:05.601837 8222 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:05.607182 8222 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.65:46683
I20260806 06:31:05.607245 8407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.65:46683 every 8 connection(s)
I20260806 06:31:05.607491 8222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
Aug 06 06:31:05 dist-test-slave-6zbq krb5kdc[8080](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997865, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.65@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:31:05.615316 8218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:05.608098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:40021 (local address 127.7.43.126:43039)
0806 06:31:05.608155 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:05.608157 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:05.608322 (+ 165us) server_negotiation.cc:407] Connection header received
0806 06:31:05.609308 (+ 986us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:05.609309 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:05.609318 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:05.609354 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:05.609869 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:05.610371 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:05.611266 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:05.611445 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:05.611486 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:05.613278 (+ 1792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:05.613280 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:05.613280 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:05.613288 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:05.614358 (+ 1070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:05.614678 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:05.614679 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:05.614679 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:05.614721 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:05.614880 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:05.614881 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:05.614881 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:05.614968 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:05.615010 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:05.615193 (+ 183us) server_negotiation.cc:299] Negotiation successful
0806 06:31:05.615246 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 06:31:05.615442 8411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:05.608053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:43039 (local address 127.7.43.65:40021)
0806 06:31:05.608217 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:05.608254 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:05.609214 (+ 960us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:05.609412 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:05.609414 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:05.609645 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:05.609814 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:05.609818 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:05.610436 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:05.610436 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:05.611206 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:05.611208 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:05.611495 (+ 287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:05.611496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:05.611581 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:05.611930 (+ 349us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:05.611937 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:05.613182 (+ 1245us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:05.614414 (+ 1232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:05.614420 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:05.614422 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:05.614613 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:05.614761 (+ 148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:05.614761 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:05.614762 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:05.614842 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:05.615007 (+ 165us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:05.615010 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:05.615103 (+ 93us) client_negotiation.cc:769] Sending connection context
0806 06:31:05.615177 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 06:31:05.615283 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":96,"threads_started":1}
I20260806 06:31:05.615873 8408 heartbeater.cc:347] Connected to a master server at 127.7.43.126:43039
I20260806 06:31:05.615947 8408 heartbeater.cc:464] Registering TS with master...
I20260806 06:31:05.616073 8408 heartbeater.cc:511] Master 127.7.43.126:43039 requested a full tablet report, sending...
I20260806 06:31:05.616505 8137 ts_manager.cc:194] Registered new tserver with Master: dcea9c51487145c1bc83fec5f9e9d794 (127.7.43.65:46683)
I20260806 06:31:05.617057 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 8222
I20260806 06:31:05.617105 8137 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.65@KRBTEST.COM'} at 127.7.43.65:40021
I20260806 06:31:05.617137 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/ts-0/wal/instance
I20260806 06:31:05.617282 7341 external_mini_cluster.cc:1428] Setting key 2b7941246c7cafb535eb86e443dac450
WARNING: no policy specified for kudu/127.7.43.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab.
Entry for principal kudu/127.7.43.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab.
I20260806 06:31:05.638388 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.66:0
--local_ip_for_outbound_sockets=127.7.43.66
--webserver_interface=127.7.43.66
--webserver_port=0
--tserver_master_addrs=127.7.43.126:43039
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab
--principal=kudu/127.7.43.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:32987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:31:05.716279 8414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:05.716450 8414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:05.716473 8414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:05.717517 8414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:05.717547 8414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:05.717571 8414 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.66
I20260806 06:31:05.719195 8414 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:32987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab
--principal=kudu/127.7.43.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.43.66
--webserver_port=0
--tserver_master_addrs=127.7.43.126:43039
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8414
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.66
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:05.719415 8414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:05.719625 8414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:05.724146 8424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:05.724164 8426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:05.724221 8414 server_base.cc:1034] running on GCE node
W20260806 06:31:05.724143 8431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:05.724562 8414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:05.724792 8414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:05.725920 8414 hybrid_clock.cc:648] HybridClock initialized: now 1785997865725906 us; error 27 us; skew 500 ppm
Aug 06 06:31:05 dist-test-slave-6zbq krb5kdc[8080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997865, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:05.728215 8414 init.cc:377] Logged in from keytab as kudu/127.7.43.66@KRBTEST.COM (short username kudu)
I20260806 06:31:05.728609 8414 webserver.cc:460] Webserver started at http://127.7.43.66:37555/ using document root <none> and password file <none>
I20260806 06:31:05.728796 8414 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:05.728842 8414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:05.728955 8414 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:05.729897 8414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/ts-1/data/instance:
uuid: "ea83efbe5fcb4b0f81cb19745c8b1908"
format_stamp: "Formatted at 2026-08-06 06:31:05 on dist-test-slave-6zbq"
server_key: "7dc52c437699f5fd54c67d59357d279f"
server_key_iv: "6ca41115ce9af589bbaa68c609e315cb"
server_key_version: "encryptionkey@0"
I20260806 06:31:05.730204 8414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/ts-1/wal/instance:
uuid: "ea83efbe5fcb4b0f81cb19745c8b1908"
format_stamp: "Formatted at 2026-08-06 06:31:05 on dist-test-slave-6zbq"
server_key: "7dc52c437699f5fd54c67d59357d279f"
server_key_iv: "6ca41115ce9af589bbaa68c609e315cb"
server_key_version: "encryptionkey@0"
I20260806 06:31:05.731436 8414 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:31:05.732053 8438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:05.732196 8414 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:31:05.732275 8414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/ts-1/wal
uuid: "ea83efbe5fcb4b0f81cb19745c8b1908"
format_stamp: "Formatted at 2026-08-06 06:31:05 on dist-test-slave-6zbq"
server_key: "7dc52c437699f5fd54c67d59357d279f"
server_key_iv: "6ca41115ce9af589bbaa68c609e315cb"
server_key_version: "encryptionkey@0"
I20260806 06:31:05.732345 8414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:05.758067 8414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:05.760059 8414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:05.760186 8414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:05.760380 8414 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:31:05.760645 8414 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:31:05.760677 8414 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:05.760704 8414 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:31:05.760722 8414 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:05.766104 8414 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.66:36875
I20260806 06:31:05.766155 8592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.66:36875 every 8 connection(s)
I20260806 06:31:05.766448 8414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
Aug 06 06:31:05 dist-test-slave-6zbq krb5kdc[8080](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997865, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.66@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:31:05.772686 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 8414
I20260806 06:31:05.772776 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/ts-1/wal/instance
I20260806 06:31:05.772938 7341 external_mini_cluster.cc:1428] Setting key 57ef06695cb3dfd77eec57731f570db5
I20260806 06:31:05.774606 8218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:05.767030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:47865 (local address 127.7.43.126:43039)
0806 06:31:05.767093 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:05.767094 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:05.767272 (+ 178us) server_negotiation.cc:407] Connection header received
0806 06:31:05.768139 (+ 867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:05.768140 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:05.768150 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:05.768207 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:05.768715 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:05.769206 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:05.770084 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:05.770268 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:05.770309 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:05.772163 (+ 1854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:05.772164 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:05.772165 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:05.772171 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:05.773373 (+ 1202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:05.773770 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:05.773772 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:05.773772 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:05.773831 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:05.774025 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:05.774025 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:05.774026 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:05.774155 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:05.774218 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:05.774437 (+ 219us) server_negotiation.cc:299] Negotiation successful
0806 06:31:05.774521 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 06:31:05.774690 8595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:05.766997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:43039 (local address 127.7.43.66:47865)
0806 06:31:05.767150 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:05.767188 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:05.768046 (+ 858us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:05.768261 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:05.768264 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:05.768481 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:05.768650 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:05.768654 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:05.769271 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:05.769272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:05.770023 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:05.770025 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:05.770319 (+ 294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:05.770320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:05.770481 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:05.770825 (+ 344us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:05.770833 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:05.772080 (+ 1247us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:05.773445 (+ 1365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:05.773450 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:05.773452 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:05.773707 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:05.773888 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:05.773889 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:05.773889 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:05.773975 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:05.774210 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:05.774212 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:05.774312 (+ 100us) client_negotiation.cc:769] Sending connection context
0806 06:31:05.774414 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 06:31:05.774526 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":83,"threads_started":1}
I20260806 06:31:05.775097 8593 heartbeater.cc:347] Connected to a master server at 127.7.43.126:43039
I20260806 06:31:05.775204 8593 heartbeater.cc:464] Registering TS with master...
I20260806 06:31:05.775374 8593 heartbeater.cc:511] Master 127.7.43.126:43039 requested a full tablet report, sending...
I20260806 06:31:05.775717 8137 ts_manager.cc:194] Registered new tserver with Master: ea83efbe5fcb4b0f81cb19745c8b1908 (127.7.43.66:36875)
I20260806 06:31:05.776067 8137 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.66@KRBTEST.COM'} at 127.7.43.66:47865
WARNING: no policy specified for kudu/127.7.43.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab.
Entry for principal kudu/127.7.43.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab.
I20260806 06:31:05.794229 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.67:0
--local_ip_for_outbound_sockets=127.7.43.67
--webserver_interface=127.7.43.67
--webserver_port=0
--tserver_master_addrs=127.7.43.126:43039
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab
--principal=kudu/127.7.43.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:32987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:31:05.871196 8601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:05.871379 8601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:05.871411 8601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:05.872519 8601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:05.872556 8601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:05.872581 8601 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.67
I20260806 06:31:05.873957 8601 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:32987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab
--principal=kudu/127.7.43.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.43.67
--webserver_port=0
--tserver_master_addrs=127.7.43.126:43039
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8601
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.67
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:05.874118 8601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:05.874284 8601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:05.878741 8608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:05.878742 8613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:05.878855 8601 server_base.cc:1034] running on GCE node
W20260806 06:31:05.878775 8610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:05.879191 8601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:05.879416 8601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:05.880548 8601 hybrid_clock.cc:648] HybridClock initialized: now 1785997865880536 us; error 31 us; skew 500 ppm
Aug 06 06:31:05 dist-test-slave-6zbq krb5kdc[8080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997865, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:05.882817 8601 init.cc:377] Logged in from keytab as kudu/127.7.43.67@KRBTEST.COM (short username kudu)
I20260806 06:31:05.883216 8601 webserver.cc:460] Webserver started at http://127.7.43.67:41571/ using document root <none> and password file <none>
I20260806 06:31:05.883435 8601 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:05.883481 8601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:05.883595 8601 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:05.884671 8601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/ts-2/data/instance:
uuid: "9516d43c4923496d9adf92bc89294483"
format_stamp: "Formatted at 2026-08-06 06:31:05 on dist-test-slave-6zbq"
server_key: "116740699ea1757e32dc560a776a6954"
server_key_iv: "98eff88649f067a1030d255e17ecde6b"
server_key_version: "encryptionkey@0"
I20260806 06:31:05.885039 8601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/ts-2/wal/instance:
uuid: "9516d43c4923496d9adf92bc89294483"
format_stamp: "Formatted at 2026-08-06 06:31:05 on dist-test-slave-6zbq"
server_key: "116740699ea1757e32dc560a776a6954"
server_key_iv: "98eff88649f067a1030d255e17ecde6b"
server_key_version: "encryptionkey@0"
I20260806 06:31:05.886332 8601 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:31:05.887010 8625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:05.887152 8601 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:31:05.887230 8601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/ts-2/wal
uuid: "9516d43c4923496d9adf92bc89294483"
format_stamp: "Formatted at 2026-08-06 06:31:05 on dist-test-slave-6zbq"
server_key: "116740699ea1757e32dc560a776a6954"
server_key_iv: "98eff88649f067a1030d255e17ecde6b"
server_key_version: "encryptionkey@0"
I20260806 06:31:05.887296 8601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:05.900870 8601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:05.902603 8601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:05.902726 8601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:05.902917 8601 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:31:05.903168 8601 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:31:05.903199 8601 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:05.903225 8601 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:31:05.903249 8601 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:05.911214 8601 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.67:40261
I20260806 06:31:05.911270 8785 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.67:40261 every 8 connection(s)
I20260806 06:31:05.911518 8601 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
Aug 06 06:31:05 dist-test-slave-6zbq krb5kdc[8080](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997865, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.67@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:31:05.918190 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 8601
I20260806 06:31:05.918282 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785997861828271-7341-0/minicluster-data/ts-2/wal/instance
I20260806 06:31:05.918454 7341 external_mini_cluster.cc:1428] Setting key 3b4d6a43b48b5f5418f67c205d40437e
I20260806 06:31:05.920885 8218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:05.912120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:52893 (local address 127.7.43.126:43039)
0806 06:31:05.912160 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:05.912162 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:05.912387 (+ 225us) server_negotiation.cc:407] Connection header received
0806 06:31:05.913530 (+ 1143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:05.913531 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:05.913543 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:05.913610 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:05.914384 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:05.915178 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:05.916337 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:05.916532 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:05.916603 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:05.918403 (+ 1800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:05.918405 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:05.918405 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:05.918412 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:05.919694 (+ 1282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:05.920035 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:05.920036 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:05.920036 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:05.920096 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:05.920306 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:05.920307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:05.920307 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:05.920418 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:05.920472 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:05.920710 (+ 238us) server_negotiation.cc:299] Negotiation successful
0806 06:31:05.920797 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 06:31:05.921036 8788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:05.912095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:43039 (local address 127.7.43.67:52893)
0806 06:31:05.912282 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:05.912313 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:05.913424 (+ 1111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:05.913680 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:05.913684 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:05.913971 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:05.914261 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:05.914268 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:05.915271 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:05.915272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:05.916233 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:05.916235 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:05.916601 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:05.916601 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:05.916688 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:05.917045 (+ 357us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:05.917052 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:05.918294 (+ 1242us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:05.919766 (+ 1472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:05.919772 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:05.919774 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:05.919970 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:05.920149 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:05.920149 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:05.920149 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:05.920245 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:05.920472 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:05.920478 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:05.920599 (+ 121us) client_negotiation.cc:769] Sending connection context
0806 06:31:05.920681 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 06:31:05.920817 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":120,"threads_started":1}
I20260806 06:31:05.921416 8786 heartbeater.cc:347] Connected to a master server at 127.7.43.126:43039
I20260806 06:31:05.921499 8786 heartbeater.cc:464] Registering TS with master...
I20260806 06:31:05.921659 8786 heartbeater.cc:511] Master 127.7.43.126:43039 requested a full tablet report, sending...
I20260806 06:31:05.921972 8137 ts_manager.cc:194] Registered new tserver with Master: 9516d43c4923496d9adf92bc89294483 (127.7.43.67:40261)
I20260806 06:31:05.922310 8137 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.67@KRBTEST.COM'} at 127.7.43.67:52893
I20260806 06:31:05.929968 7341 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:31:05.937078 8218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:05.931865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48232 (local address 127.7.43.126:43039)
0806 06:31:05.931915 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:05.931917 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:05.932009 (+ 92us) server_negotiation.cc:407] Connection header received
0806 06:31:05.932050 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:05.932051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:05.932058 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:05.932127 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:05.932592 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:05.933022 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:05.933647 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:05.933816 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:05.933856 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:05.934874 (+ 1018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:05.934875 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:05.934875 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:05.934880 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:05.936050 (+ 1170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:05.936405 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:05.936406 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:05.936406 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:05.936458 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:05.936613 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:05.936614 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:05.936614 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:05.936741 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:05.936827 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:05.936934 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 06:31:05.936999 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 06:31:05.939361 8137 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48232:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:31:05.941185 8137 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:31:05.947973 8794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:05.943746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.67:40261 (local address 127.0.0.1:40332)
0806 06:31:05.943901 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:05.943920 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:05.944083 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:05.944716 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:05.944717 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:05.944719 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:31:05.944794 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:05.944796 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:05.945955 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:05.945956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:05.946777 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:05.946780 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:05.947693 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:05.947694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:05.947779 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:05.947794 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 06:31:05.947857 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 06:31:05.947906 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":2,"thread_start_us":50,"threads_started":1}
I20260806 06:31:05.948163 8802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:05.944222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50780 (local address 127.7.43.66:36875)
0806 06:31:05.944468 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:05.944473 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:05.944483 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:31:05.944521 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:05.944522 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:05.944534 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:05.944597 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:31:05.944697 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:05.945565 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:05.947216 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:05.947719 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:05.947783 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:05.947838 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:05.947948 (+ 110us) server_negotiation.cc:299] Negotiation successful
0806 06:31:05.947989 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":139,"threads_started":1}
I20260806 06:31:05.948217 8798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:05.943879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40332 (local address 127.7.43.67:40261)
0806 06:31:05.944512 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:05.944516 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:05.944525 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:31:05.944556 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:05.944558 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:05.944572 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:05.944645 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:31:05.944867 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:05.945809 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:05.946850 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:05.947595 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:05.947672 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:05.947725 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:05.948119 (+ 394us) server_negotiation.cc:299] Negotiation successful
0806 06:31:05.948155 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":74,"threads_started":1}
I20260806 06:31:05.948388 8801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:05.943985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52850 (local address 127.7.43.65:46683)
0806 06:31:05.944141 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:05.944145 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:05.944156 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:31:05.944233 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:05.944235 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:05.944251 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:05.944348 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:31:05.944523 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:05.945616 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:05.947019 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:05.947701 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:05.947768 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:05.947819 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:05.948191 (+ 372us) server_negotiation.cc:299] Negotiation successful
0806 06:31:05.948229 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":77,"threads_started":1}
I20260806 06:31:05.948086 8796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:05.943746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.66:36875 (local address 127.0.0.1:50780)
0806 06:31:05.943888 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:05.943920 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:05.943982 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:05.944601 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:05.944602 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:05.944604 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:31:05.944639 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:05.944641 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:05.945660 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:05.945661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:05.947149 (+ 1488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:05.947150 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:05.947802 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:05.947803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:05.947874 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:05.947878 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:31:05.947929 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 06:31:05.947972 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":46,"thread_start_us":51,"threads_started":1}
I20260806 06:31:05.948135 8797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:05.943773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:46683 (local address 127.0.0.1:52850)
0806 06:31:05.943954 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:05.943961 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:05.943999 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:05.944366 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:05.944369 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:05.944371 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:31:05.944433 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:05.944435 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:05.945709 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:05.945710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:05.946942 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:05.946945 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:05.947849 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:05.947850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:05.947952 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:05.947956 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:31:05.947998 (+ 42us) client_negotiation.cc:240] Negotiation successful
0806 06:31:05.948030 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":34,"thread_start_us":59,"threads_started":1}
I20260806 06:31:05.949067 8689 tablet_service.cc:1467] Processing CreateTablet for tablet 128273fb8b324faaa7e73692a3e81811 (DEFAULT_TABLE table=test-table [id=7945e793190b46e980deb9bceb76f7a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:31:05.949139 8311 tablet_service.cc:1467] Processing CreateTablet for tablet 128273fb8b324faaa7e73692a3e81811 (DEFAULT_TABLE table=test-table [id=7945e793190b46e980deb9bceb76f7a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:31:05.949397 8689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 128273fb8b324faaa7e73692a3e81811. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:05.949420 8311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 128273fb8b324faaa7e73692a3e81811. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:05.950810 8500 tablet_service.cc:1467] Processing CreateTablet for tablet 128273fb8b324faaa7e73692a3e81811 (DEFAULT_TABLE table=test-table [id=7945e793190b46e980deb9bceb76f7a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:31:05.951088 8500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 128273fb8b324faaa7e73692a3e81811. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:05.952783 8803 tablet_bootstrap.cc:492] T 128273fb8b324faaa7e73692a3e81811 P 9516d43c4923496d9adf92bc89294483: Bootstrap starting.
I20260806 06:31:05.953526 8803 tablet_bootstrap.cc:654] T 128273fb8b324faaa7e73692a3e81811 P 9516d43c4923496d9adf92bc89294483: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:05.954402 8803 log.cc:824] T 128273fb8b324faaa7e73692a3e81811 P 9516d43c4923496d9adf92bc89294483: Log is configured to *not* fsync() on all Append() calls
I20260806 06:31:05.954448 8806 tablet_bootstrap.cc:492] T 128273fb8b324faaa7e73692a3e81811 P ea83efbe5fcb4b0f81cb19745c8b1908: Bootstrap starting.
I20260806 06:31:05.954741 8805 tablet_bootstrap.cc:492] T 128273fb8b324faaa7e73692a3e81811 P dcea9c51487145c1bc83fec5f9e9d794: Bootstrap starting.
I20260806 06:31:05.955099 8806 tablet_bootstrap.cc:654] T 128273fb8b324faaa7e73692a3e81811 P ea83efbe5fcb4b0f81cb19745c8b1908: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:05.955152 8803 tablet_bootstrap.cc:492] T 128273fb8b324faaa7e73692a3e81811 P 9516d43c4923496d9adf92bc89294483: No bootstrap required, opened a new log
I20260806 06:31:05.955206 8803 ts_tablet_manager.cc:1397] T 128273fb8b324faaa7e73692a3e81811 P 9516d43c4923496d9adf92bc89294483: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 06:31:05.955492 8805 tablet_bootstrap.cc:654] T 128273fb8b324faaa7e73692a3e81811 P dcea9c51487145c1bc83fec5f9e9d794: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:05.955628 8806 log.cc:824] T 128273fb8b324faaa7e73692a3e81811 P ea83efbe5fcb4b0f81cb19745c8b1908: Log is configured to *not* fsync() on all Append() calls
I20260806 06:31:05.956197 8805 log.cc:824] T 128273fb8b324faaa7e73692a3e81811 P dcea9c51487145c1bc83fec5f9e9d794: Log is configured to *not* fsync() on all Append() calls
I20260806 06:31:05.956432 8806 tablet_bootstrap.cc:492] T 128273fb8b324faaa7e73692a3e81811 P ea83efbe5fcb4b0f81cb19745c8b1908: No bootstrap required, opened a new log
I20260806 06:31:05.956501 8806 ts_tablet_manager.cc:1397] T 128273fb8b324faaa7e73692a3e81811 P ea83efbe5fcb4b0f81cb19745c8b1908: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:31:05.956816 8805 tablet_bootstrap.cc:492] T 128273fb8b324faaa7e73692a3e81811 P dcea9c51487145c1bc83fec5f9e9d794: No bootstrap required, opened a new log
I20260806 06:31:05.956876 8805 ts_tablet_manager.cc:1397] T 128273fb8b324faaa7e73692a3e81811 P dcea9c51487145c1bc83fec5f9e9d794: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:31:05.957592 8803 raft_consensus.cc:348] T 128273fb8b324faaa7e73692a3e81811 P 9516d43c4923496d9adf92bc89294483 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9516d43c4923496d9adf92bc89294483" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 40261 } } peers { permanent_uuid: "ea83efbe5fcb4b0f81cb19745c8b1908" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 36875 } } peers { permanent_uuid: "dcea9c51487145c1bc83fec5f9e9d794" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 46683 } }
I20260806 06:31:05.957768 8803 raft_consensus.cc:374] T 128273fb8b324faaa7e73692a3e81811 P 9516d43c4923496d9adf92bc89294483 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:05.957821 8803 raft_consensus.cc:729] T 128273fb8b324faaa7e73692a3e81811 P 9516d43c4923496d9adf92bc89294483 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9516d43c4923496d9adf92bc89294483, State: Initialized, Role: FOLLOWER
I20260806 06:31:05.957937 8803 consensus_queue.cc:260] T 128273fb8b324faaa7e73692a3e81811 P 9516d43c4923496d9adf92bc89294483 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9516d43c4923496d9adf92bc89294483" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 40261 } } peers { permanent_uuid: "ea83efbe5fcb4b0f81cb19745c8b1908" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 36875 } } peers { permanent_uuid: "dcea9c51487145c1bc83fec5f9e9d794" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 46683 } }
I20260806 06:31:05.958202 8803 ts_tablet_manager.cc:1428] T 128273fb8b324faaa7e73692a3e81811 P 9516d43c4923496d9adf92bc89294483: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 06:31:05.958402 8786 heartbeater.cc:503] Master 127.7.43.126:43039 was elected leader, sending a full tablet report...
I20260806 06:31:05.958770 8806 raft_consensus.cc:348] T 128273fb8b324faaa7e73692a3e81811 P ea83efbe5fcb4b0f81cb19745c8b1908 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9516d43c4923496d9adf92bc89294483" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 40261 } } peers { permanent_uuid: "ea83efbe5fcb4b0f81cb19745c8b1908" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 36875 } } peers { permanent_uuid: "dcea9c51487145c1bc83fec5f9e9d794" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 46683 } }
I20260806 06:31:05.958930 8806 raft_consensus.cc:374] T 128273fb8b324faaa7e73692a3e81811 P ea83efbe5fcb4b0f81cb19745c8b1908 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:05.958971 8806 raft_consensus.cc:729] T 128273fb8b324faaa7e73692a3e81811 P ea83efbe5fcb4b0f81cb19745c8b1908 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea83efbe5fcb4b0f81cb19745c8b1908, State: Initialized, Role: FOLLOWER
I20260806 06:31:05.959079 8806 consensus_queue.cc:260] T 128273fb8b324faaa7e73692a3e81811 P ea83efbe5fcb4b0f81cb19745c8b1908 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9516d43c4923496d9adf92bc89294483" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 40261 } } peers { permanent_uuid: "ea83efbe5fcb4b0f81cb19745c8b1908" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 36875 } } peers { permanent_uuid: "dcea9c51487145c1bc83fec5f9e9d794" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 46683 } }
I20260806 06:31:05.958971 8805 raft_consensus.cc:348] T 128273fb8b324faaa7e73692a3e81811 P dcea9c51487145c1bc83fec5f9e9d794 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9516d43c4923496d9adf92bc89294483" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 40261 } } peers { permanent_uuid: "ea83efbe5fcb4b0f81cb19745c8b1908" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 36875 } } peers { permanent_uuid: "dcea9c51487145c1bc83fec5f9e9d794" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 46683 } }
I20260806 06:31:05.959203 8805 raft_consensus.cc:374] T 128273fb8b324faaa7e73692a3e81811 P dcea9c51487145c1bc83fec5f9e9d794 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:05.959259 8805 raft_consensus.cc:729] T 128273fb8b324faaa7e73692a3e81811 P dcea9c51487145c1bc83fec5f9e9d794 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcea9c51487145c1bc83fec5f9e9d794, State: Initialized, Role: FOLLOWER
I20260806 06:31:05.959357 8806 ts_tablet_manager.cc:1428] T 128273fb8b324faaa7e73692a3e81811 P ea83efbe5fcb4b0f81cb19745c8b1908: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260806 06:31:05.959376 8805 consensus_queue.cc:260] T 128273fb8b324faaa7e73692a3e81811 P dcea9c51487145c1bc83fec5f9e9d794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9516d43c4923496d9adf92bc89294483" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 40261 } } peers { permanent_uuid: "ea83efbe5fcb4b0f81cb19745c8b1908" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 36875 } } peers { permanent_uuid: "dcea9c51487145c1bc83fec5f9e9d794" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 46683 } }
I20260806 06:31:05.959457 8593 heartbeater.cc:503] Master 127.7.43.126:43039 was elected leader, sending a full tablet report...
I20260806 06:31:05.959728 8805 ts_tablet_manager.cc:1428] T 128273fb8b324faaa7e73692a3e81811 P dcea9c51487145c1bc83fec5f9e9d794: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 06:31:05.959867 8408 heartbeater.cc:503] Master 127.7.43.126:43039 was elected leader, sending a full tablet report...
W20260806 06:31:06.016935 8594 tablet.cc:2369] T 128273fb8b324faaa7e73692a3e81811 P ea83efbe5fcb4b0f81cb19745c8b1908: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 06:31:06.108158 8409 tablet.cc:2369] T 128273fb8b324faaa7e73692a3e81811 P dcea9c51487145c1bc83fec5f9e9d794: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 06:31:06.162122 8787 tablet.cc:2369] T 128273fb8b324faaa7e73692a3e81811 P 9516d43c4923496d9adf92bc89294483: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:31:06.341262 8811 raft_consensus.cc:482] T 128273fb8b324faaa7e73692a3e81811 P 9516d43c4923496d9adf92bc89294483 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:31:06.341388 8811 raft_consensus.cc:504] T 128273fb8b324faaa7e73692a3e81811 P 9516d43c4923496d9adf92bc89294483 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9516d43c4923496d9adf92bc89294483" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 40261 } } peers { permanent_uuid: "ea83efbe5fcb4b0f81cb19745c8b1908" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 36875 } } peers { permanent_uuid: "dcea9c51487145c1bc83fec5f9e9d794" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 46683 } }
I20260806 06:31:06.341991 8811 leader_election.cc:288] T 128273fb8b324faaa7e73692a3e81811 P 9516d43c4923496d9adf92bc89294483 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ea83efbe5fcb4b0f81cb19745c8b1908 (127.7.43.66:36875), dcea9c51487145c1bc83fec5f9e9d794 (127.7.43.65:46683)
I20260806 06:31:06.344708 8788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:06.342148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.66:36875 (local address 127.7.43.67:34613)
0806 06:31:06.342197 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:06.342208 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:06.342250 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:06.342421 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:06.342422 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:06.342424 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:31:06.342494 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:06.342497 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:06.343177 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:06.343178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:06.343949 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:06.343952 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:06.344473 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:06.344474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:06.344553 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:06.344557 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:31:06.344604 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 06:31:06.344637 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260806 06:31:06.344729 8802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:06.342187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:34613 (local address 127.7.43.66:36875)
0806 06:31:06.342232 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:06.342234 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:06.342245 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:31:06.342351 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:06.342352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:06.342363 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:06.342407 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:31:06.342579 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:06.343102 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:06.344037 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:06.344402 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:06.344452 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:06.344462 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:06.344622 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 06:31:06.344660 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:31:06.344777 8801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:06.342252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:34723 (local address 127.7.43.65:46683)
0806 06:31:06.342307 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:06.342309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:06.342505 (+ 196us) server_negotiation.cc:407] Connection header received
0806 06:31:06.342536 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:06.342537 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:06.342550 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:06.342597 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:31:06.342737 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:06.343202 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:06.344114 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:06.344469 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:06.344536 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:06.344546 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:06.344678 (+ 132us) server_negotiation.cc:299] Negotiation successful
0806 06:31:06.344709 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 06:31:06.344827 8816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:06.342155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:46683 (local address 127.7.43.67:34723)
0806 06:31:06.342431 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:06.342440 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:06.342497 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:06.342611 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:06.342612 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:06.342613 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:31:06.342679 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:06.342681 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:06.343276 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:06.343276 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:06.344030 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:06.344032 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:06.344543 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:06.344544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:06.344606 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:06.344611 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:31:06.344662 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 06:31:06.344697 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"mutex_wait_us":44,"thread_start_us":110,"threads_started":1}
I20260806 06:31:06.345218 8529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "128273fb8b324faaa7e73692a3e81811" candidate_uuid: "9516d43c4923496d9adf92bc89294483" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea83efbe5fcb4b0f81cb19745c8b1908" is_pre_election: true
I20260806 06:31:06.345222 8341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "128273fb8b324faaa7e73692a3e81811" candidate_uuid: "9516d43c4923496d9adf92bc89294483" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcea9c51487145c1bc83fec5f9e9d794" is_pre_election: true
I20260806 06:31:06.345393 8341 raft_consensus.cc:2454] T 128273fb8b324faaa7e73692a3e81811 P dcea9c51487145c1bc83fec5f9e9d794 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9516d43c4923496d9adf92bc89294483 in term 0.
I20260806 06:31:06.345402 8529 raft_consensus.cc:2454] T 128273fb8b324faaa7e73692a3e81811 P ea83efbe5fcb4b0f81cb19745c8b1908 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9516d43c4923496d9adf92bc89294483 in term 0.
I20260806 06:31:06.345717 8630 leader_election.cc:302] T 128273fb8b324faaa7e73692a3e81811 P 9516d43c4923496d9adf92bc89294483 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9516d43c4923496d9adf92bc89294483, ea83efbe5fcb4b0f81cb19745c8b1908; no voters:
I20260806 06:31:06.345882 8811 raft_consensus.cc:2781] T 128273fb8b324faaa7e73692a3e81811 P 9516d43c4923496d9adf92bc89294483 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:31:06.345933 8811 raft_consensus.cc:482] T 128273fb8b324faaa7e73692a3e81811 P 9516d43c4923496d9adf92bc89294483 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:31:06.345948 8811 raft_consensus.cc:3037] T 128273fb8b324faaa7e73692a3e81811 P 9516d43c4923496d9adf92bc89294483 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:06.346616 8811 raft_consensus.cc:504] T 128273fb8b324faaa7e73692a3e81811 P 9516d43c4923496d9adf92bc89294483 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9516d43c4923496d9adf92bc89294483" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 40261 } } peers { permanent_uuid: "ea83efbe5fcb4b0f81cb19745c8b1908" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 36875 } } peers { permanent_uuid: "dcea9c51487145c1bc83fec5f9e9d794" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 46683 } }
I20260806 06:31:06.346728 8811 leader_election.cc:288] T 128273fb8b324faaa7e73692a3e81811 P 9516d43c4923496d9adf92bc89294483 [CANDIDATE]: Term 1 election: Requested vote from peers ea83efbe5fcb4b0f81cb19745c8b1908 (127.7.43.66:36875), dcea9c51487145c1bc83fec5f9e9d794 (127.7.43.65:46683)
I20260806 06:31:06.346894 8529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "128273fb8b324faaa7e73692a3e81811" candidate_uuid: "9516d43c4923496d9adf92bc89294483" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea83efbe5fcb4b0f81cb19745c8b1908"
I20260806 06:31:06.346964 8529 raft_consensus.cc:3037] T 128273fb8b324faaa7e73692a3e81811 P ea83efbe5fcb4b0f81cb19745c8b1908 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:06.347008 8341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "128273fb8b324faaa7e73692a3e81811" candidate_uuid: "9516d43c4923496d9adf92bc89294483" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcea9c51487145c1bc83fec5f9e9d794"
I20260806 06:31:06.347095 8341 raft_consensus.cc:3037] T 128273fb8b324faaa7e73692a3e81811 P dcea9c51487145c1bc83fec5f9e9d794 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:06.347662 8529 raft_consensus.cc:2454] T 128273fb8b324faaa7e73692a3e81811 P ea83efbe5fcb4b0f81cb19745c8b1908 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9516d43c4923496d9adf92bc89294483 in term 1.
I20260806 06:31:06.347824 8630 leader_election.cc:302] T 128273fb8b324faaa7e73692a3e81811 P 9516d43c4923496d9adf92bc89294483 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9516d43c4923496d9adf92bc89294483, ea83efbe5fcb4b0f81cb19745c8b1908; no voters:
I20260806 06:31:06.347901 8811 raft_consensus.cc:2781] T 128273fb8b324faaa7e73692a3e81811 P 9516d43c4923496d9adf92bc89294483 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:31:06.347929 8341 raft_consensus.cc:2454] T 128273fb8b324faaa7e73692a3e81811 P dcea9c51487145c1bc83fec5f9e9d794 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9516d43c4923496d9adf92bc89294483 in term 1.
I20260806 06:31:06.348049 8811 raft_consensus.cc:686] T 128273fb8b324faaa7e73692a3e81811 P 9516d43c4923496d9adf92bc89294483 [term 1 LEADER]: Becoming Leader. State: Replica: 9516d43c4923496d9adf92bc89294483, State: Running, Role: LEADER
I20260806 06:31:06.348201 8811 consensus_queue.cc:237] T 128273fb8b324faaa7e73692a3e81811 P 9516d43c4923496d9adf92bc89294483 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9516d43c4923496d9adf92bc89294483" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 40261 } } peers { permanent_uuid: "ea83efbe5fcb4b0f81cb19745c8b1908" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 36875 } } peers { permanent_uuid: "dcea9c51487145c1bc83fec5f9e9d794" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 46683 } }
I20260806 06:31:06.349116 8136 catalog_manager.cc:5495] T 128273fb8b324faaa7e73692a3e81811 P 9516d43c4923496d9adf92bc89294483 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9516d43c4923496d9adf92bc89294483 (127.7.43.67). New cstate: current_term: 1 leader_uuid: "9516d43c4923496d9adf92bc89294483" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9516d43c4923496d9adf92bc89294483" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 40261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ea83efbe5fcb4b0f81cb19745c8b1908" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 36875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dcea9c51487145c1bc83fec5f9e9d794" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 46683 } health_report { overall_health: UNKNOWN } } }
Aug 06 06:31:06 dist-test-slave-6zbq krb5kdc[8080](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.43.65@KRBTEST.COM
I20260806 06:31:06.387650 8801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:06.381131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52858 (local address 127.7.43.65:46683)
0806 06:31:06.381179 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:06.381182 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:06.381293 (+ 111us) server_negotiation.cc:407] Connection header received
0806 06:31:06.381355 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:06.381355 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:06.381365 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:06.381411 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:06.381917 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:06.382385 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:06.383039 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:06.383229 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:06.383280 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:06.385324 (+ 2044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:06.385333 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:06.385333 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:06.385340 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:06.386623 (+ 1283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:06.386943 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:06.386945 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:06.386946 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:06.387005 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:06.387166 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:06.387167 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:06.387167 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:06.387294 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:06.387418 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:06.387491 (+ 73us) server_negotiation.cc:299] Negotiation successful
0806 06:31:06.387565 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
Aug 06 06:31:06 dist-test-slave-6zbq krb5kdc[8080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997866, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 06:31:06 dist-test-slave-6zbq krb5kdc[8080](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997866, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.43.65@KRBTEST.COM
I20260806 06:31:06.409749 8801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:06.403894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52870 (local address 127.7.43.65:46683)
0806 06:31:06.403951 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:06.403953 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:06.404010 (+ 57us) server_negotiation.cc:407] Connection header received
0806 06:31:06.404069 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:06.404070 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:06.404079 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:06.404124 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:06.404576 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:06.405008 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:06.405661 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:06.405840 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:06.405891 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:06.407798 (+ 1907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:06.407799 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:06.407800 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:06.407806 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:06.408786 (+ 980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:06.409065 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:06.409066 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:06.409066 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:06.409116 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:06.409282 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:06.409282 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:06.409283 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:06.409370 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:06.409449 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:06.409596 (+ 147us) server_negotiation.cc:299] Negotiation successful
0806 06:31:06.409666 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
Aug 06 06:31:06 dist-test-slave-6zbq krb5kdc[8080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 06:31:06 dist-test-slave-6zbq krb5kdc[8080](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.43.65@KRBTEST.COM
I20260806 06:31:06.430873 8801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:06.425182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52874 (local address 127.7.43.65:46683)
0806 06:31:06.425238 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:06.425240 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:06.425305 (+ 65us) server_negotiation.cc:407] Connection header received
0806 06:31:06.425365 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:06.425365 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:06.425374 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:06.425420 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:06.425865 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:06.426289 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:06.426983 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:06.427157 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:06.427208 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:06.428953 (+ 1745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:06.428954 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:06.428955 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:06.428961 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:06.430040 (+ 1079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:06.430305 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:06.430305 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:06.430305 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:06.430367 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:06.430506 (+ 139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:06.430506 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:06.430507 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:06.430586 (+ 79us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:06.430652 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:06.430768 (+ 116us) server_negotiation.cc:299] Negotiation successful
0806 06:31:06.430817 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
W20260806 06:31:06.431555 8274 tablet_service.cc:2908] Rejecting scan request for tablet 128273fb8b324faaa7e73692a3e81811: Uninitialized: clean time has not yet been initialized
I20260806 06:31:06.432332 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 8222
I20260806 06:31:06.445916 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 8414
I20260806 06:31:06.457594 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 8601
I20260806 06:31:06.463662 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 8097
2026-08-06T06:31:06Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2356 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:31:06 dist-test-slave-6zbq krb5kdc[8837](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:31:06 dist-test-slave-6zbq krb5kdc[8837](info): set up 2 sockets
Aug 06 06:31:06 dist-test-slave-6zbq krb5kdc[8837](info): commencing operation
krb5kdc: starting...
W20260806 06:31:07.569974 7341 mini_kdc.cc:121] Time spent starting KDC: real 1.082s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Aug 06 06:31:07 dist-test-slave-6zbq krb5kdc[8837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:31:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:31:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.43.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
I20260806 06:31:07.680869 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:36741
--webserver_interface=127.7.43.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:39041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.43.126:36741
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:31:07.764878 8854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:07.765049 8854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:07.765069 8854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:07.766155 8854 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:31:07.766202 8854 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 06:31:07.766213 8854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:07.766238 8854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:07.766261 8854 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:31:07.766281 8854 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:31:07.767726 8854 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:39041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.43.126:36741
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:36741
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.43.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8854
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:07.767933 8854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:07.768119 8854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:07.772450 8860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:07.772457 8864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:07.772490 8859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:07.772686 8854 server_base.cc:1034] running on GCE node
I20260806 06:31:07.772908 8854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:07.773154 8854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:07.774300 8854 hybrid_clock.cc:648] HybridClock initialized: now 1785997867774285 us; error 28 us; skew 500 ppm
Aug 06 06:31:07 dist-test-slave-6zbq krb5kdc[8837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997867, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:07.776530 8854 init.cc:377] Logged in from keytab as kudu/127.7.43.126@KRBTEST.COM (short username kudu)
I20260806 06:31:07.776904 8854 webserver.cc:460] Webserver started at http://127.7.43.126:38383/ using document root <none> and password file <none>
I20260806 06:31:07.777107 8854 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:07.777151 8854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:07.777269 8854 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:07.778198 8854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/master-0/data/instance:
uuid: "1b10352f33a04b8fa06d09a309070f3b"
format_stamp: "Formatted at 2026-08-06 06:31:07 on dist-test-slave-6zbq"
server_key: "bc9d8816588eef25257a6ad47cd47528"
server_key_iv: "3f955d98f4d71ce8b56eefb1f79085b7"
server_key_version: "encryptionkey@0"
I20260806 06:31:07.778540 8854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/master-0/wal/instance:
uuid: "1b10352f33a04b8fa06d09a309070f3b"
format_stamp: "Formatted at 2026-08-06 06:31:07 on dist-test-slave-6zbq"
server_key: "bc9d8816588eef25257a6ad47cd47528"
server_key_iv: "3f955d98f4d71ce8b56eefb1f79085b7"
server_key_version: "encryptionkey@0"
I20260806 06:31:07.779687 8854 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:31:07.780263 8873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:07.780403 8854 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:07.780474 8854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "1b10352f33a04b8fa06d09a309070f3b"
format_stamp: "Formatted at 2026-08-06 06:31:07 on dist-test-slave-6zbq"
server_key: "bc9d8816588eef25257a6ad47cd47528"
server_key_iv: "3f955d98f4d71ce8b56eefb1f79085b7"
server_key_version: "encryptionkey@0"
I20260806 06:31:07.780534 8854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:07.795698 8854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:07.797168 8854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:07.797289 8854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:07.801625 8854 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.126:36741
I20260806 06:31:07.801684 8959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.126:36741 every 8 connection(s)
I20260806 06:31:07.801937 8854 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:31:07.802573 8964 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:07.804610 8964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b10352f33a04b8fa06d09a309070f3b: Bootstrap starting.
I20260806 06:31:07.805125 8964 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b10352f33a04b8fa06d09a309070f3b: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:07.805364 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 8854
I20260806 06:31:07.805397 8964 log.cc:824] T 00000000000000000000000000000000 P 1b10352f33a04b8fa06d09a309070f3b: Log is configured to *not* fsync() on all Append() calls
I20260806 06:31:07.805441 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/master-0/wal/instance
I20260806 06:31:07.805579 7341 external_mini_cluster.cc:1428] Setting key 96b7a23c72a4c50f0f5040fe56fe5f02
I20260806 06:31:07.806103 8964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b10352f33a04b8fa06d09a309070f3b: No bootstrap required, opened a new log
I20260806 06:31:07.808094 8964 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b10352f33a04b8fa06d09a309070f3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b10352f33a04b8fa06d09a309070f3b" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36741 } }
I20260806 06:31:07.808259 8964 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b10352f33a04b8fa06d09a309070f3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:07.808293 8964 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b10352f33a04b8fa06d09a309070f3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b10352f33a04b8fa06d09a309070f3b, State: Initialized, Role: FOLLOWER
I20260806 06:31:07.808408 8964 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b10352f33a04b8fa06d09a309070f3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b10352f33a04b8fa06d09a309070f3b" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36741 } }
I20260806 06:31:07.808477 8964 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b10352f33a04b8fa06d09a309070f3b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:31:07.808511 8964 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b10352f33a04b8fa06d09a309070f3b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:31:07.808552 8964 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b10352f33a04b8fa06d09a309070f3b [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:07.809141 8964 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b10352f33a04b8fa06d09a309070f3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b10352f33a04b8fa06d09a309070f3b" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36741 } }
I20260806 06:31:07.809253 8964 leader_election.cc:302] T 00000000000000000000000000000000 P 1b10352f33a04b8fa06d09a309070f3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b10352f33a04b8fa06d09a309070f3b; no voters:
I20260806 06:31:07.809423 8964 leader_election.cc:288] T 00000000000000000000000000000000 P 1b10352f33a04b8fa06d09a309070f3b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:31:07.809468 8970 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b10352f33a04b8fa06d09a309070f3b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:31:07.809603 8970 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b10352f33a04b8fa06d09a309070f3b [term 1 LEADER]: Becoming Leader. State: Replica: 1b10352f33a04b8fa06d09a309070f3b, State: Running, Role: LEADER
I20260806 06:31:07.809650 8964 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b10352f33a04b8fa06d09a309070f3b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:31:07.809690 8970 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b10352f33a04b8fa06d09a309070f3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b10352f33a04b8fa06d09a309070f3b" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36741 } }
I20260806 06:31:07.810001 8970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b10352f33a04b8fa06d09a309070f3b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b10352f33a04b8fa06d09a309070f3b. Latest consensus state: current_term: 1 leader_uuid: "1b10352f33a04b8fa06d09a309070f3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b10352f33a04b8fa06d09a309070f3b" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36741 } } }
I20260806 06:31:07.810086 8970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b10352f33a04b8fa06d09a309070f3b [sys.catalog]: This master's current role is: LEADER
I20260806 06:31:07.810012 8972 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b10352f33a04b8fa06d09a309070f3b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1b10352f33a04b8fa06d09a309070f3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b10352f33a04b8fa06d09a309070f3b" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36741 } } }
I20260806 06:31:07.810283 8972 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b10352f33a04b8fa06d09a309070f3b [sys.catalog]: This master's current role is: LEADER
I20260806 06:31:07.811290 8981 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:31:07.811646 8981 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:31:07.812822 8981 catalog_manager.cc:1347] Generated new cluster ID: f27e5ce3e69f4f13be118c6045542991
I20260806 06:31:07.812868 8981 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:31:07 dist-test-slave-6zbq krb5kdc[8837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:31:07.816996 8968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:07.805991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47490 (local address 127.7.43.126:36741)
0806 06:31:07.806154 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:07.806163 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:07.806195 (+ 32us) server_negotiation.cc:407] Connection header received
0806 06:31:07.806646 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:07.806655 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:07.806721 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:07.806860 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:07.807378 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:07.809202 (+ 1824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:07.810260 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:07.810889 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:07.810992 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:07.813631 (+ 2639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:07.813634 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:07.813635 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:07.813645 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:07.815496 (+ 1851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:07.815888 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:07.815890 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:07.815891 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:07.815980 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:07.816169 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:07.816170 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:07.816170 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:07.816415 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:07.816545 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:07.816672 (+ 127us) server_negotiation.cc:299] Negotiation successful
0806 06:31:07.816798 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":79,"threads_started":1}
I20260806 06:31:07.818483 8981 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:31:07.818918 8981 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:31:07.827322 8981 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1b10352f33a04b8fa06d09a309070f3b: Generated new TSK 0
I20260806 06:31:07.827466 8981 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.43.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab.
Entry for principal kudu/127.7.43.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab.
I20260806 06:31:07.850723 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.65:0
--local_ip_for_outbound_sockets=127.7.43.65
--webserver_interface=127.7.43.65
--webserver_port=0
--tserver_master_addrs=127.7.43.126:36741
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab
--principal=kudu/127.7.43.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:39041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:31:07.958371 8998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:07.958561 8998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:07.958583 8998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:07.959693 8998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:07.959731 8998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:07.959750 8998 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 06:31:07.959766 8998 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.65
I20260806 06:31:07.961143 8998 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:39041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab
--principal=kudu/127.7.43.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.43.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.43.126:36741
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8998
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.65
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:07.961354 8998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:07.961552 8998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:31:07.966065 8998 server_base.cc:1034] running on GCE node
W20260806 06:31:07.966058 9008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:07.966190 9010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:07.966282 9012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:07.966513 8998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:07.966740 8998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:07.967880 8998 hybrid_clock.cc:648] HybridClock initialized: now 1785997867967862 us; error 31 us; skew 500 ppm
Aug 06 06:31:07 dist-test-slave-6zbq krb5kdc[8837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997867, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:07.970121 8998 init.cc:377] Logged in from keytab as kudu/127.7.43.65@KRBTEST.COM (short username kudu)
I20260806 06:31:07.970569 8998 webserver.cc:460] Webserver started at http://127.7.43.65:46705/ using document root <none> and password file <none>
I20260806 06:31:07.970758 8998 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:07.970806 8998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:07.970921 8998 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:07.971863 8998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/ts-0/data/instance:
uuid: "a255e41a83d6499ea839047c801780cb"
format_stamp: "Formatted at 2026-08-06 06:31:07 on dist-test-slave-6zbq"
server_key: "a7fd74b4c05f77d06327e5c96260c4ab"
server_key_iv: "5ab26ad5f7d24a40288d9d106d8fabb1"
server_key_version: "encryptionkey@0"
I20260806 06:31:07.972184 8998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/ts-0/wal/instance:
uuid: "a255e41a83d6499ea839047c801780cb"
format_stamp: "Formatted at 2026-08-06 06:31:07 on dist-test-slave-6zbq"
server_key: "a7fd74b4c05f77d06327e5c96260c4ab"
server_key_iv: "5ab26ad5f7d24a40288d9d106d8fabb1"
server_key_version: "encryptionkey@0"
I20260806 06:31:07.973378 8998 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 06:31:07.974035 9021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:07.974169 8998 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:07.974251 8998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/ts-0/wal
uuid: "a255e41a83d6499ea839047c801780cb"
format_stamp: "Formatted at 2026-08-06 06:31:07 on dist-test-slave-6zbq"
server_key: "a7fd74b4c05f77d06327e5c96260c4ab"
server_key_iv: "5ab26ad5f7d24a40288d9d106d8fabb1"
server_key_version: "encryptionkey@0"
I20260806 06:31:07.974390 8998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:07.988790 8998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:07.990585 8998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:07.990710 8998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:07.991189 8998 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:31:07.991227 8998 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:07.991255 8998 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:31:07.991289 8998 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 06:31:07 dist-test-slave-6zbq krb5kdc[8837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997867, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.65@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:31:07.998021 8998 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.65:44971
I20260806 06:31:07.998096 9187 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.65:44971 every 8 connection(s)
I20260806 06:31:07.998425 8998 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
I20260806 06:31:08.000030 8968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:07.991710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:34975 (local address 127.7.43.126:36741)
0806 06:31:07.991784 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:07.991788 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:07.991956 (+ 168us) server_negotiation.cc:407] Connection header received
0806 06:31:07.992399 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:07.992401 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:07.992414 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:07.992487 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:07.993085 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:07.993830 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:07.995306 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:07.995531 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:07.995592 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:07.997539 (+ 1947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:07.997540 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:07.997541 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:07.997548 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:07.999043 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:07.999375 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:07.999376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:07.999376 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:07.999424 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:07.999588 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:07.999588 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:07.999589 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:07.999679 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:07.999721 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:07.999911 (+ 190us) server_negotiation.cc:299] Negotiation successful
0806 06:31:07.999968 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 06:31:08.000181 9040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:07.991671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:36741 (local address 127.7.43.65:34975)
0806 06:31:07.991836 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:07.991874 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:07.992312 (+ 438us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:07.992568 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:07.992571 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:07.992818 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:07.992994 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:07.992998 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:07.993932 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:07.993934 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:07.995221 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:07.995224 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:07.995665 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:07.995666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:07.995767 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:07.996187 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:07.996194 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:07.997453 (+ 1259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:07.999118 (+ 1665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:07.999121 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:07.999123 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:07.999322 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:07.999466 (+ 144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:07.999467 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:07.999467 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:07.999548 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:07.999718 (+ 170us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:07.999721 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:07.999820 (+ 99us) client_negotiation.cc:769] Sending connection context
0806 06:31:07.999893 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 06:31:08.000003 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":87,"threads_started":1}
I20260806 06:31:08.000645 9188 heartbeater.cc:347] Connected to a master server at 127.7.43.126:36741
I20260806 06:31:08.000735 9188 heartbeater.cc:464] Registering TS with master...
I20260806 06:31:08.000912 9188 heartbeater.cc:511] Master 127.7.43.126:36741 requested a full tablet report, sending...
I20260806 06:31:08.001456 8894 ts_manager.cc:194] Registered new tserver with Master: a255e41a83d6499ea839047c801780cb (127.7.43.65:44971)
I20260806 06:31:08.002241 8894 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.65@KRBTEST.COM'} at 127.7.43.65:34975
I20260806 06:31:08.005815 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 8998
I20260806 06:31:08.005901 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/ts-0/wal/instance
I20260806 06:31:08.006067 7341 external_mini_cluster.cc:1428] Setting key 8dd75e9eea755dfa490dcfe3484aee81
I20260806 06:31:08.006642 9194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:08.001687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:36741 (local address 127.7.43.65:60473)
0806 06:31:08.001884 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:08.001895 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:08.001958 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:08.002066 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:08.002066 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:08.002257 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:08.002320 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:08.002322 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.002971 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.002972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:08.003471 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:08.003473 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.003790 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.003791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:08.003868 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:08.004198 (+ 330us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:08.004202 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:08.004449 (+ 247us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:08.005638 (+ 1189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:08.005639 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:08.005639 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:08.005797 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:08.005974 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:08.005974 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:08.005975 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:08.006035 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:08.006310 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:08.006311 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:08.006385 (+ 74us) client_negotiation.cc:769] Sending connection context
0806 06:31:08.006451 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 06:31:08.006548 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":131,"threads_started":1}
I20260806 06:31:08.006641 8968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:08.001731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:60473 (local address 127.7.43.126:36741)
0806 06:31:08.001763 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:08.001765 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:08.001951 (+ 186us) server_negotiation.cc:407] Connection header received
0806 06:31:08.002011 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:08.002011 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:08.002019 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:08.002061 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:08.002403 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.002881 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.003542 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.003726 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.003779 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:08.004525 (+ 746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:08.004526 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:08.004526 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:08.004533 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:08.005565 (+ 1032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:08.005855 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:08.005856 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:08.005857 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:08.005907 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:08.006102 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:08.006103 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:08.006104 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:08.006237 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:08.006310 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:08.006492 (+ 182us) server_negotiation.cc:299] Negotiation successful
0806 06:31:08.006558 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
WARNING: no policy specified for kudu/127.7.43.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab.
Entry for principal kudu/127.7.43.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab.
I20260806 06:31:08.028702 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.66:0
--local_ip_for_outbound_sockets=127.7.43.66
--webserver_interface=127.7.43.66
--webserver_port=0
--tserver_master_addrs=127.7.43.126:36741
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab
--principal=kudu/127.7.43.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:39041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:31:08.108554 9198 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:08.108734 9198 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:08.108757 9198 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:08.109849 9198 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:08.109886 9198 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:08.109905 9198 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 06:31:08.109928 9198 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.66
I20260806 06:31:08.111347 9198 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:39041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab
--principal=kudu/127.7.43.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.43.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.43.126:36741
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9198
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.66
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:08.111529 9198 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:08.111691 9198 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:08.116238 9207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:08.116206 9205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:08.116343 9198 server_base.cc:1034] running on GCE node
W20260806 06:31:08.116226 9210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:08.116652 9198 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:08.116871 9198 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:08.118007 9198 hybrid_clock.cc:648] HybridClock initialized: now 1785997868117996 us; error 26 us; skew 500 ppm
Aug 06 06:31:08 dist-test-slave-6zbq krb5kdc[8837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997868, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:08.120318 9198 init.cc:377] Logged in from keytab as kudu/127.7.43.66@KRBTEST.COM (short username kudu)
I20260806 06:31:08.120711 9198 webserver.cc:460] Webserver started at http://127.7.43.66:44085/ using document root <none> and password file <none>
I20260806 06:31:08.120918 9198 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:08.120963 9198 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:08.121083 9198 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:08.122032 9198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/ts-1/data/instance:
uuid: "40a5a24cba3b489885e1df5b3febda4d"
format_stamp: "Formatted at 2026-08-06 06:31:08 on dist-test-slave-6zbq"
server_key: "df1bcd682795a4f2a3b18778490a5088"
server_key_iv: "13849925424f148fa8a744b3205d66ab"
server_key_version: "encryptionkey@0"
I20260806 06:31:08.122406 9198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/ts-1/wal/instance:
uuid: "40a5a24cba3b489885e1df5b3febda4d"
format_stamp: "Formatted at 2026-08-06 06:31:08 on dist-test-slave-6zbq"
server_key: "df1bcd682795a4f2a3b18778490a5088"
server_key_iv: "13849925424f148fa8a744b3205d66ab"
server_key_version: "encryptionkey@0"
I20260806 06:31:08.123616 9198 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:31:08.124289 9220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:08.124434 9198 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:31:08.124528 9198 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/ts-1/wal
uuid: "40a5a24cba3b489885e1df5b3febda4d"
format_stamp: "Formatted at 2026-08-06 06:31:08 on dist-test-slave-6zbq"
server_key: "df1bcd682795a4f2a3b18778490a5088"
server_key_iv: "13849925424f148fa8a744b3205d66ab"
server_key_version: "encryptionkey@0"
I20260806 06:31:08.124591 9198 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:08.131047 9198 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:08.132555 9198 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:08.132675 9198 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:08.133129 9198 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:31:08.133168 9198 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:08.133194 9198 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:31:08.133211 9198 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 06:31:08 dist-test-slave-6zbq krb5kdc[8837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997868, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.66@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:31:08.140025 9198 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.66:37701
I20260806 06:31:08.140127 9383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.66:37701 every 8 connection(s)
I20260806 06:31:08.140373 9198 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
I20260806 06:31:08.142373 8968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:08.133466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:44941 (local address 127.7.43.126:36741)
0806 06:31:08.133529 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:08.133532 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:08.133722 (+ 190us) server_negotiation.cc:407] Connection header received
0806 06:31:08.134299 (+ 577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:08.134300 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:08.134310 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:08.134383 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:08.135150 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.135651 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.137053 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.137242 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.137310 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:08.139923 (+ 2613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:08.139926 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:08.139926 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:08.139933 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:08.141332 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:08.141667 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:08.141668 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:08.141668 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:08.141712 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:08.141876 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:08.141877 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:08.141877 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:08.141967 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:08.142010 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:08.142222 (+ 212us) server_negotiation.cc:299] Negotiation successful
0806 06:31:08.142286 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 06:31:08.142596 9231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:08.133433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:36741 (local address 127.7.43.66:44941)
0806 06:31:08.133585 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:08.133637 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:08.134201 (+ 564us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:08.134452 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:08.134456 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:08.134786 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:08.135052 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:08.135058 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.135734 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.135735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:08.136977 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:08.136981 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.137333 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.137334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:08.137473 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:08.137881 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:08.137890 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:08.139818 (+ 1928us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:08.141409 (+ 1591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:08.141412 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:08.141414 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:08.141611 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:08.141750 (+ 139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:08.141751 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:08.141751 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:08.141836 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:08.142013 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:08.142016 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:08.142129 (+ 113us) client_negotiation.cc:769] Sending connection context
0806 06:31:08.142203 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 06:31:08.142311 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":82,"threads_started":1}
I20260806 06:31:08.142670 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 9198
I20260806 06:31:08.142750 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/ts-1/wal/instance
I20260806 06:31:08.142951 7341 external_mini_cluster.cc:1428] Setting key f531e7420dbf8ed8899bad5263207aa2
I20260806 06:31:08.143072 9384 heartbeater.cc:347] Connected to a master server at 127.7.43.126:36741
I20260806 06:31:08.143175 9384 heartbeater.cc:464] Registering TS with master...
I20260806 06:31:08.143357 9384 heartbeater.cc:511] Master 127.7.43.126:36741 requested a full tablet report, sending...
I20260806 06:31:08.143743 8894 ts_manager.cc:194] Registered new tserver with Master: 40a5a24cba3b489885e1df5b3febda4d (127.7.43.66:37701)
I20260806 06:31:08.144157 8894 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.66@KRBTEST.COM'} at 127.7.43.66:44941
I20260806 06:31:08.150383 9394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:08.145239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:36741 (local address 127.7.43.66:33909)
0806 06:31:08.145361 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:08.145372 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:08.145432 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:08.145540 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:08.145540 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:08.145720 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:08.145776 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:08.145778 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.146415 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.146416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:08.146904 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:08.146906 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.147326 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.147326 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:08.147370 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:08.147741 (+ 371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:08.147746 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:08.147994 (+ 248us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:08.149458 (+ 1464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:08.149459 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:08.149460 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:08.149605 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:08.149783 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:08.149783 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:08.149783 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:08.149826 (+ 43us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:08.150129 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:08.150130 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:08.150165 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 06:31:08.150211 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 06:31:08.150280 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":67,"threads_started":1}
I20260806 06:31:08.150441 8968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:08.145324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:33909 (local address 127.7.43.126:36741)
0806 06:31:08.145364 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:08.145366 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:08.145418 (+ 52us) server_negotiation.cc:407] Connection header received
0806 06:31:08.145481 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:08.145481 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:08.145488 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:08.145526 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:08.145839 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.146323 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.146980 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.147244 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.147328 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:08.148072 (+ 744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:08.148074 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:08.148074 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:08.148081 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:08.149387 (+ 1306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:08.149669 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:08.149670 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:08.149670 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:08.149726 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:08.149874 (+ 148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:08.149874 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:08.149874 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:08.150052 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:08.150114 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:08.150254 (+ 140us) server_negotiation.cc:299] Negotiation successful
0806 06:31:08.150333 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
WARNING: no policy specified for kudu/127.7.43.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab.
Entry for principal kudu/127.7.43.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab.
I20260806 06:31:08.166612 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.67:0
--local_ip_for_outbound_sockets=127.7.43.67
--webserver_interface=127.7.43.67
--webserver_port=0
--tserver_master_addrs=127.7.43.126:36741
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab
--principal=kudu/127.7.43.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:39041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:31:08.245528 9396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:08.245713 9396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:08.245734 9396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:08.246840 9396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:08.246874 9396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:08.246892 9396 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 06:31:08.246908 9396 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.67
I20260806 06:31:08.248246 9396 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:39041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab
--principal=kudu/127.7.43.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.43.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.43.126:36741
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9396
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.67
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:08.248418 9396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:08.248581 9396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:08.253079 9401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:08.253099 9407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:08.253209 9396 server_base.cc:1034] running on GCE node
W20260806 06:31:08.253099 9403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:08.253430 9396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:08.253611 9396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:08.254733 9396 hybrid_clock.cc:648] HybridClock initialized: now 1785997868254720 us; error 28 us; skew 500 ppm
Aug 06 06:31:08 dist-test-slave-6zbq krb5kdc[8837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997868, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:08.256973 9396 init.cc:377] Logged in from keytab as kudu/127.7.43.67@KRBTEST.COM (short username kudu)
I20260806 06:31:08.257355 9396 webserver.cc:460] Webserver started at http://127.7.43.67:42297/ using document root <none> and password file <none>
I20260806 06:31:08.257525 9396 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:08.257562 9396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:08.257647 9396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:08.258595 9396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/ts-2/data/instance:
uuid: "6a6fc9246e5d40f3a293032c65a7f6a2"
format_stamp: "Formatted at 2026-08-06 06:31:08 on dist-test-slave-6zbq"
server_key: "b172b07241d7dea9e7d2ed8f6634b490"
server_key_iv: "40096a90374f65fbb476bece6c5066dd"
server_key_version: "encryptionkey@0"
I20260806 06:31:08.258880 9396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/ts-2/wal/instance:
uuid: "6a6fc9246e5d40f3a293032c65a7f6a2"
format_stamp: "Formatted at 2026-08-06 06:31:08 on dist-test-slave-6zbq"
server_key: "b172b07241d7dea9e7d2ed8f6634b490"
server_key_iv: "40096a90374f65fbb476bece6c5066dd"
server_key_version: "encryptionkey@0"
I20260806 06:31:08.259954 9396 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:31:08.260653 9416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:08.260800 9396 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:31:08.260880 9396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/ts-2/wal
uuid: "6a6fc9246e5d40f3a293032c65a7f6a2"
format_stamp: "Formatted at 2026-08-06 06:31:08 on dist-test-slave-6zbq"
server_key: "b172b07241d7dea9e7d2ed8f6634b490"
server_key_iv: "40096a90374f65fbb476bece6c5066dd"
server_key_version: "encryptionkey@0"
I20260806 06:31:08.260938 9396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:08.297884 9396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:08.299862 9396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:08.299989 9396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:08.300431 9396 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:31:08.300479 9396 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:08.300508 9396 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:31:08.300524 9396 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 06:31:08 dist-test-slave-6zbq krb5kdc[8837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997868, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.67@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:31:08.307176 9396 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.67:35147
I20260806 06:31:08.307224 9584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.67:35147 every 8 connection(s)
I20260806 06:31:08.307544 9396 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
I20260806 06:31:08.309376 8968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:08.300833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:52147 (local address 127.7.43.126:36741)
0806 06:31:08.300897 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:08.300900 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:08.301133 (+ 233us) server_negotiation.cc:407] Connection header received
0806 06:31:08.301587 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:08.301588 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:08.301600 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:08.301646 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:08.302426 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.303168 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.304115 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.304410 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.304490 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:08.306909 (+ 2419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:08.306911 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:08.306912 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:08.306921 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:08.308334 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:08.308663 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:08.308664 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:08.308664 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:08.308710 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:08.308879 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:08.308880 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:08.308880 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:08.308969 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:08.309040 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:08.309254 (+ 214us) server_negotiation.cc:299] Negotiation successful
0806 06:31:08.309314 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 06:31:08.309513 9427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:08.300835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:36741 (local address 127.7.43.67:52147)
0806 06:31:08.300996 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:08.301029 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:08.301492 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:08.301722 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:08.301726 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:08.302027 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:08.302289 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:08.302295 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.303267 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.303268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:08.304038 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:08.304040 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.304514 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.304514 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:08.304615 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:08.305048 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:08.305057 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:08.306816 (+ 1759us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:08.308401 (+ 1585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:08.308404 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:08.308405 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:08.308606 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:08.308754 (+ 148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:08.308754 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:08.308755 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:08.308838 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:08.309034 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:08.309039 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:08.309148 (+ 109us) client_negotiation.cc:769] Sending connection context
0806 06:31:08.309229 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 06:31:08.309337 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":78,"threads_started":1}
I20260806 06:31:08.309967 9585 heartbeater.cc:347] Connected to a master server at 127.7.43.126:36741
I20260806 06:31:08.310046 9585 heartbeater.cc:464] Registering TS with master...
I20260806 06:31:08.310254 9585 heartbeater.cc:511] Master 127.7.43.126:36741 requested a full tablet report, sending...
I20260806 06:31:08.310644 8895 ts_manager.cc:194] Registered new tserver with Master: 6a6fc9246e5d40f3a293032c65a7f6a2 (127.7.43.67:35147)
I20260806 06:31:08.310875 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 9396
I20260806 06:31:08.310958 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/ts-2/wal/instance
I20260806 06:31:08.311126 8895 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.67@KRBTEST.COM'} at 127.7.43.67:52147
I20260806 06:31:08.311167 7341 external_mini_cluster.cc:1428] Setting key 9b589a586bfdf483cdf8c7a54c1e9eba
I20260806 06:31:08.311962 7341 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:31:08.316284 8968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:08.311476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:51967 (local address 127.7.43.126:36741)
0806 06:31:08.311523 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:08.311525 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:08.311533 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:31:08.311551 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:08.311552 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:08.311562 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:08.311619 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:08.311952 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.312426 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.313239 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.313415 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.313467 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:08.314289 (+ 822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:08.314290 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:08.314290 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:08.314297 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:08.315386 (+ 1089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:08.315700 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:08.315701 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:08.315701 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:08.315754 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:08.315927 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:08.315927 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:08.315928 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:08.316006 (+ 78us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:08.316047 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:08.316178 (+ 131us) server_negotiation.cc:299] Negotiation successful
0806 06:31:08.316232 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 06:31:08.316543 9591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:08.311070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:36741 (local address 127.7.43.67:51967)
0806 06:31:08.311236 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:08.311248 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:08.311297 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:08.311626 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:08.311626 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:08.311822 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:08.311882 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:08.311884 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.312521 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.312522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:08.313160 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:08.313163 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.313479 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.313480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:08.313546 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:08.313879 (+ 333us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:08.313884 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:08.314208 (+ 324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:08.315458 (+ 1250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:08.315459 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:08.315459 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:08.315637 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:08.315812 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:08.315812 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:08.315813 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:08.315874 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:08.316053 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:08.316053 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:08.316095 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 06:31:08.316159 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 06:31:08.316254 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":62,"threads_started":1}
Aug 06 06:31:08 dist-test-slave-6zbq krb5kdc[8837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997868, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 06:31:08 dist-test-slave-6zbq krb5kdc[8837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997868, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:31:08.334081 8968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:08.328555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47500 (local address 127.7.43.126:36741)
0806 06:31:08.328599 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:08.328601 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:08.328608 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:31:08.328641 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:08.328642 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:08.328649 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:08.328684 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:08.329160 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.329588 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.330213 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.330412 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.330452 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:08.332281 (+ 1829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:08.332282 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:08.332282 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:08.332288 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:08.333245 (+ 957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:08.333530 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:08.333531 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:08.333531 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:08.333571 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:08.333724 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:08.333725 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:08.333726 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:08.333831 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:08.333910 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:08.333978 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 06:31:08.334030 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 06:31:08.335457 8895 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47500:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:31:08.336930 8895 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:31:08.342749 9607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:08.338930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:44971 (local address 127.0.0.1:39230)
0806 06:31:08.339141 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:08.339151 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:08.339201 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:08.339534 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:08.339538 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:08.339541 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:31:08.339607 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:08.339608 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.340550 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.340551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:08.341618 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:08.341620 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.342513 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.342514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:08.342570 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:08.342579 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 06:31:08.342634 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 06:31:08.342681 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"mutex_wait_us":5,"thread_start_us":73,"threads_started":1}
I20260806 06:31:08.343004 9606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:08.338929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.67:35147 (local address 127.0.0.1:48436)
0806 06:31:08.339269 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:08.339276 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:08.339309 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:08.339878 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:08.339879 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:08.339881 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:31:08.339919 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:08.339920 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.341037 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.341038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:08.341863 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:08.341866 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.342780 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.342781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:08.342838 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:08.342843 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:31:08.342897 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 06:31:08.342934 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"mutex_wait_us":9,"thread_start_us":72,"threads_started":1}
I20260806 06:31:08.343084 9610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:08.339225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48436 (local address 127.7.43.67:35147)
0806 06:31:08.339631 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:08.339634 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:08.339642 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:31:08.339672 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:08.339674 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:08.339686 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:08.339758 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:31:08.339993 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.340979 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.341930 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.342627 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.342696 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:08.342744 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:08.342927 (+ 183us) server_negotiation.cc:299] Negotiation successful
0806 06:31:08.342965 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":72,"threads_started":1}
I20260806 06:31:08.343015 9609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:08.339139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39230 (local address 127.7.43.65:44971)
0806 06:31:08.339333 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:08.339338 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:08.339348 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:31:08.339387 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:08.339390 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:08.339406 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:08.339493 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:31:08.339707 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.340470 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.341716 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.342385 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.342444 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:08.342484 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:08.342668 (+ 184us) server_negotiation.cc:299] Negotiation successful
0806 06:31:08.342742 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":112,"threads_started":1}
I20260806 06:31:08.343233 9605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:08.338920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.66:37701 (local address 127.0.0.1:56956)
0806 06:31:08.339057 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:08.339089 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:08.339162 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:08.339551 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:08.339552 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:08.339553 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:31:08.339604 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:08.339605 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.340967 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.340968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:08.342004 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:08.342006 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.342999 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.343017 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:08.343104 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:08.343109 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:31:08.343138 (+ 29us) client_negotiation.cc:240] Negotiation successful
0806 06:31:08.343174 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":48,"threads_started":1}
I20260806 06:31:08.343724 9486 tablet_service.cc:1467] Processing CreateTablet for tablet 390e5b5258874d5ea7b828880607edfa (DEFAULT_TABLE table=test-table [id=b85f7326a44f45b68f667796936e1cac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:31:08.343804 9096 tablet_service.cc:1467] Processing CreateTablet for tablet 390e5b5258874d5ea7b828880607edfa (DEFAULT_TABLE table=test-table [id=b85f7326a44f45b68f667796936e1cac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:31:08.344074 9486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 390e5b5258874d5ea7b828880607edfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:08.344122 9096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 390e5b5258874d5ea7b828880607edfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:08.347424 9613 tablet_bootstrap.cc:492] T 390e5b5258874d5ea7b828880607edfa P 6a6fc9246e5d40f3a293032c65a7f6a2: Bootstrap starting.
I20260806 06:31:08.347779 9608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:08.339059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56956 (local address 127.7.43.66:37701)
0806 06:31:08.339393 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:08.339397 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:08.339405 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:31:08.339442 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:08.339444 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:08.339457 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:08.339528 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:31:08.339734 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.340880 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.342081 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.342808 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.347551 (+ 4743us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:08.347617 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:08.347644 (+ 27us) server_negotiation.cc:299] Negotiation successful
0806 06:31:08.347682 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":86,"threads_started":1}
I20260806 06:31:08.347833 9614 tablet_bootstrap.cc:492] T 390e5b5258874d5ea7b828880607edfa P a255e41a83d6499ea839047c801780cb: Bootstrap starting.
I20260806 06:31:08.348285 9289 tablet_service.cc:1467] Processing CreateTablet for tablet 390e5b5258874d5ea7b828880607edfa (DEFAULT_TABLE table=test-table [id=b85f7326a44f45b68f667796936e1cac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:31:08.348568 9289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 390e5b5258874d5ea7b828880607edfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:08.348603 9614 tablet_bootstrap.cc:654] T 390e5b5258874d5ea7b828880607edfa P a255e41a83d6499ea839047c801780cb: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:08.348603 9613 tablet_bootstrap.cc:654] T 390e5b5258874d5ea7b828880607edfa P 6a6fc9246e5d40f3a293032c65a7f6a2: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:08.348935 9613 log.cc:824] T 390e5b5258874d5ea7b828880607edfa P 6a6fc9246e5d40f3a293032c65a7f6a2: Log is configured to *not* fsync() on all Append() calls
I20260806 06:31:08.349090 9614 log.cc:824] T 390e5b5258874d5ea7b828880607edfa P a255e41a83d6499ea839047c801780cb: Log is configured to *not* fsync() on all Append() calls
I20260806 06:31:08.349684 9613 tablet_bootstrap.cc:492] T 390e5b5258874d5ea7b828880607edfa P 6a6fc9246e5d40f3a293032c65a7f6a2: No bootstrap required, opened a new log
I20260806 06:31:08.349736 9613 ts_tablet_manager.cc:1397] T 390e5b5258874d5ea7b828880607edfa P 6a6fc9246e5d40f3a293032c65a7f6a2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:31:08.350943 9614 tablet_bootstrap.cc:492] T 390e5b5258874d5ea7b828880607edfa P a255e41a83d6499ea839047c801780cb: No bootstrap required, opened a new log
I20260806 06:31:08.351017 9614 ts_tablet_manager.cc:1397] T 390e5b5258874d5ea7b828880607edfa P a255e41a83d6499ea839047c801780cb: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 06:31:08.351429 9620 tablet_bootstrap.cc:492] T 390e5b5258874d5ea7b828880607edfa P 40a5a24cba3b489885e1df5b3febda4d: Bootstrap starting.
I20260806 06:31:08.351745 9613 raft_consensus.cc:348] T 390e5b5258874d5ea7b828880607edfa P 6a6fc9246e5d40f3a293032c65a7f6a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40a5a24cba3b489885e1df5b3febda4d" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37701 } } peers { permanent_uuid: "6a6fc9246e5d40f3a293032c65a7f6a2" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 35147 } } peers { permanent_uuid: "a255e41a83d6499ea839047c801780cb" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 44971 } }
I20260806 06:31:08.351915 9613 raft_consensus.cc:374] T 390e5b5258874d5ea7b828880607edfa P 6a6fc9246e5d40f3a293032c65a7f6a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:08.351951 9613 raft_consensus.cc:729] T 390e5b5258874d5ea7b828880607edfa P 6a6fc9246e5d40f3a293032c65a7f6a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a6fc9246e5d40f3a293032c65a7f6a2, State: Initialized, Role: FOLLOWER
I20260806 06:31:08.352003 9620 tablet_bootstrap.cc:654] T 390e5b5258874d5ea7b828880607edfa P 40a5a24cba3b489885e1df5b3febda4d: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:08.352072 9613 consensus_queue.cc:260] T 390e5b5258874d5ea7b828880607edfa P 6a6fc9246e5d40f3a293032c65a7f6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40a5a24cba3b489885e1df5b3febda4d" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37701 } } peers { permanent_uuid: "6a6fc9246e5d40f3a293032c65a7f6a2" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 35147 } } peers { permanent_uuid: "a255e41a83d6499ea839047c801780cb" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 44971 } }
I20260806 06:31:08.352353 9613 ts_tablet_manager.cc:1428] T 390e5b5258874d5ea7b828880607edfa P 6a6fc9246e5d40f3a293032c65a7f6a2: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 06:31:08.352459 9620 log.cc:824] T 390e5b5258874d5ea7b828880607edfa P 40a5a24cba3b489885e1df5b3febda4d: Log is configured to *not* fsync() on all Append() calls
I20260806 06:31:08.352527 9585 heartbeater.cc:503] Master 127.7.43.126:36741 was elected leader, sending a full tablet report...
I20260806 06:31:08.353417 9620 tablet_bootstrap.cc:492] T 390e5b5258874d5ea7b828880607edfa P 40a5a24cba3b489885e1df5b3febda4d: No bootstrap required, opened a new log
I20260806 06:31:08.353488 9620 ts_tablet_manager.cc:1397] T 390e5b5258874d5ea7b828880607edfa P 40a5a24cba3b489885e1df5b3febda4d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:31:08.353947 9614 raft_consensus.cc:348] T 390e5b5258874d5ea7b828880607edfa P a255e41a83d6499ea839047c801780cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40a5a24cba3b489885e1df5b3febda4d" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37701 } } peers { permanent_uuid: "6a6fc9246e5d40f3a293032c65a7f6a2" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 35147 } } peers { permanent_uuid: "a255e41a83d6499ea839047c801780cb" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 44971 } }
I20260806 06:31:08.354108 9614 raft_consensus.cc:374] T 390e5b5258874d5ea7b828880607edfa P a255e41a83d6499ea839047c801780cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:08.354157 9614 raft_consensus.cc:729] T 390e5b5258874d5ea7b828880607edfa P a255e41a83d6499ea839047c801780cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a255e41a83d6499ea839047c801780cb, State: Initialized, Role: FOLLOWER
I20260806 06:31:08.354272 9614 consensus_queue.cc:260] T 390e5b5258874d5ea7b828880607edfa P a255e41a83d6499ea839047c801780cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40a5a24cba3b489885e1df5b3febda4d" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37701 } } peers { permanent_uuid: "6a6fc9246e5d40f3a293032c65a7f6a2" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 35147 } } peers { permanent_uuid: "a255e41a83d6499ea839047c801780cb" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 44971 } }
I20260806 06:31:08.354547 9614 ts_tablet_manager.cc:1428] T 390e5b5258874d5ea7b828880607edfa P a255e41a83d6499ea839047c801780cb: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260806 06:31:08.354632 9188 heartbeater.cc:503] Master 127.7.43.126:36741 was elected leader, sending a full tablet report...
I20260806 06:31:08.355839 9620 raft_consensus.cc:348] T 390e5b5258874d5ea7b828880607edfa P 40a5a24cba3b489885e1df5b3febda4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40a5a24cba3b489885e1df5b3febda4d" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37701 } } peers { permanent_uuid: "6a6fc9246e5d40f3a293032c65a7f6a2" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 35147 } } peers { permanent_uuid: "a255e41a83d6499ea839047c801780cb" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 44971 } }
I20260806 06:31:08.355954 9620 raft_consensus.cc:374] T 390e5b5258874d5ea7b828880607edfa P 40a5a24cba3b489885e1df5b3febda4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:08.355973 9620 raft_consensus.cc:729] T 390e5b5258874d5ea7b828880607edfa P 40a5a24cba3b489885e1df5b3febda4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40a5a24cba3b489885e1df5b3febda4d, State: Initialized, Role: FOLLOWER
I20260806 06:31:08.356050 9620 consensus_queue.cc:260] T 390e5b5258874d5ea7b828880607edfa P 40a5a24cba3b489885e1df5b3febda4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40a5a24cba3b489885e1df5b3febda4d" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37701 } } peers { permanent_uuid: "6a6fc9246e5d40f3a293032c65a7f6a2" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 35147 } } peers { permanent_uuid: "a255e41a83d6499ea839047c801780cb" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 44971 } }
I20260806 06:31:08.356246 9620 ts_tablet_manager.cc:1428] T 390e5b5258874d5ea7b828880607edfa P 40a5a24cba3b489885e1df5b3febda4d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:31:08.356350 9384 heartbeater.cc:503] Master 127.7.43.126:36741 was elected leader, sending a full tablet report...
W20260806 06:31:08.391175 9385 tablet.cc:2369] T 390e5b5258874d5ea7b828880607edfa P 40a5a24cba3b489885e1df5b3febda4d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:31:08.406724 9626 raft_consensus.cc:482] T 390e5b5258874d5ea7b828880607edfa P a255e41a83d6499ea839047c801780cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:31:08.406816 9626 raft_consensus.cc:504] T 390e5b5258874d5ea7b828880607edfa P a255e41a83d6499ea839047c801780cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40a5a24cba3b489885e1df5b3febda4d" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37701 } } peers { permanent_uuid: "6a6fc9246e5d40f3a293032c65a7f6a2" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 35147 } } peers { permanent_uuid: "a255e41a83d6499ea839047c801780cb" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 44971 } }
I20260806 06:31:08.407136 9626 leader_election.cc:288] T 390e5b5258874d5ea7b828880607edfa P a255e41a83d6499ea839047c801780cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 40a5a24cba3b489885e1df5b3febda4d (127.7.43.66:37701), 6a6fc9246e5d40f3a293032c65a7f6a2 (127.7.43.67:35147)
I20260806 06:31:08.409806 9040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:08.407301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.67:35147 (local address 127.7.43.65:60923)
0806 06:31:08.407352 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:08.407365 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:08.407415 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:08.407573 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:08.407574 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:08.407576 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:31:08.407641 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:08.407642 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.408228 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.408230 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:08.408988 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:08.408991 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.409564 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.409566 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:08.409637 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:08.409640 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:31:08.409684 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 06:31:08.409740 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27,"mutex_wait_us":29}
I20260806 06:31:08.409803 9610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:08.407370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:60923 (local address 127.7.43.67:35147)
0806 06:31:08.407410 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:08.407412 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:08.407420 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:31:08.407502 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:08.407503 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:08.407515 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:08.407560 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:31:08.407701 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.408159 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.409064 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.409480 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.409540 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:08.409551 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:08.409698 (+ 147us) server_negotiation.cc:299] Negotiation successful
0806 06:31:08.409732 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 06:31:08.409881 9608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:08.407439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:57103 (local address 127.7.43.66:37701)
0806 06:31:08.407487 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:08.407489 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:08.407583 (+ 94us) server_negotiation.cc:407] Connection header received
0806 06:31:08.407642 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:08.407643 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:08.407655 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:08.407700 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:31:08.407833 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.408281 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.409145 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.409551 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.409616 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:08.409626 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:08.409788 (+ 162us) server_negotiation.cc:299] Negotiation successful
0806 06:31:08.409824 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 06:31:08.409897 9629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:08.407360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.66:37701 (local address 127.7.43.65:57103)
0806 06:31:08.407515 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:08.407523 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:08.407579 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:08.407708 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:08.407708 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:08.407710 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:31:08.407780 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:08.407782 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.408350 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.408351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:08.409068 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:08.409070 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.409626 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.409628 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:08.409699 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:08.409702 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:31:08.409777 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 06:31:08.409818 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":104,"threads_started":1}
I20260806 06:31:08.410187 9515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "390e5b5258874d5ea7b828880607edfa" candidate_uuid: "a255e41a83d6499ea839047c801780cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a6fc9246e5d40f3a293032c65a7f6a2" is_pre_election: true
I20260806 06:31:08.410238 9318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "390e5b5258874d5ea7b828880607edfa" candidate_uuid: "a255e41a83d6499ea839047c801780cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40a5a24cba3b489885e1df5b3febda4d" is_pre_election: true
I20260806 06:31:08.410359 9515 raft_consensus.cc:2454] T 390e5b5258874d5ea7b828880607edfa P 6a6fc9246e5d40f3a293032c65a7f6a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a255e41a83d6499ea839047c801780cb in term 0.
I20260806 06:31:08.410408 9318 raft_consensus.cc:2454] T 390e5b5258874d5ea7b828880607edfa P 40a5a24cba3b489885e1df5b3febda4d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a255e41a83d6499ea839047c801780cb in term 0.
I20260806 06:31:08.410638 9028 leader_election.cc:302] T 390e5b5258874d5ea7b828880607edfa P a255e41a83d6499ea839047c801780cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 40a5a24cba3b489885e1df5b3febda4d, 6a6fc9246e5d40f3a293032c65a7f6a2, a255e41a83d6499ea839047c801780cb; no voters:
I20260806 06:31:08.410765 9626 raft_consensus.cc:2781] T 390e5b5258874d5ea7b828880607edfa P a255e41a83d6499ea839047c801780cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:31:08.410823 9626 raft_consensus.cc:482] T 390e5b5258874d5ea7b828880607edfa P a255e41a83d6499ea839047c801780cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:31:08.410840 9626 raft_consensus.cc:3037] T 390e5b5258874d5ea7b828880607edfa P a255e41a83d6499ea839047c801780cb [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:08.411734 9626 raft_consensus.cc:504] T 390e5b5258874d5ea7b828880607edfa P a255e41a83d6499ea839047c801780cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40a5a24cba3b489885e1df5b3febda4d" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37701 } } peers { permanent_uuid: "6a6fc9246e5d40f3a293032c65a7f6a2" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 35147 } } peers { permanent_uuid: "a255e41a83d6499ea839047c801780cb" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 44971 } }
I20260806 06:31:08.411844 9626 leader_election.cc:288] T 390e5b5258874d5ea7b828880607edfa P a255e41a83d6499ea839047c801780cb [CANDIDATE]: Term 1 election: Requested vote from peers 40a5a24cba3b489885e1df5b3febda4d (127.7.43.66:37701), 6a6fc9246e5d40f3a293032c65a7f6a2 (127.7.43.67:35147)
I20260806 06:31:08.411983 9318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "390e5b5258874d5ea7b828880607edfa" candidate_uuid: "a255e41a83d6499ea839047c801780cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40a5a24cba3b489885e1df5b3febda4d"
I20260806 06:31:08.412050 9318 raft_consensus.cc:3037] T 390e5b5258874d5ea7b828880607edfa P 40a5a24cba3b489885e1df5b3febda4d [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:08.412024 9515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "390e5b5258874d5ea7b828880607edfa" candidate_uuid: "a255e41a83d6499ea839047c801780cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a6fc9246e5d40f3a293032c65a7f6a2"
I20260806 06:31:08.412099 9515 raft_consensus.cc:3037] T 390e5b5258874d5ea7b828880607edfa P 6a6fc9246e5d40f3a293032c65a7f6a2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:08.412825 9318 raft_consensus.cc:2454] T 390e5b5258874d5ea7b828880607edfa P 40a5a24cba3b489885e1df5b3febda4d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a255e41a83d6499ea839047c801780cb in term 1.
I20260806 06:31:08.413000 9515 raft_consensus.cc:2454] T 390e5b5258874d5ea7b828880607edfa P 6a6fc9246e5d40f3a293032c65a7f6a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a255e41a83d6499ea839047c801780cb in term 1.
I20260806 06:31:08.413048 9028 leader_election.cc:302] T 390e5b5258874d5ea7b828880607edfa P a255e41a83d6499ea839047c801780cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40a5a24cba3b489885e1df5b3febda4d, a255e41a83d6499ea839047c801780cb; no voters:
I20260806 06:31:08.413139 9626 raft_consensus.cc:2781] T 390e5b5258874d5ea7b828880607edfa P a255e41a83d6499ea839047c801780cb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:31:08.413280 9626 raft_consensus.cc:686] T 390e5b5258874d5ea7b828880607edfa P a255e41a83d6499ea839047c801780cb [term 1 LEADER]: Becoming Leader. State: Replica: a255e41a83d6499ea839047c801780cb, State: Running, Role: LEADER
I20260806 06:31:08.413368 9626 consensus_queue.cc:237] T 390e5b5258874d5ea7b828880607edfa P a255e41a83d6499ea839047c801780cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40a5a24cba3b489885e1df5b3febda4d" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37701 } } peers { permanent_uuid: "6a6fc9246e5d40f3a293032c65a7f6a2" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 35147 } } peers { permanent_uuid: "a255e41a83d6499ea839047c801780cb" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 44971 } }
I20260806 06:31:08.414108 8895 catalog_manager.cc:5495] T 390e5b5258874d5ea7b828880607edfa P a255e41a83d6499ea839047c801780cb reported cstate change: term changed from 0 to 1, leader changed from <none> to a255e41a83d6499ea839047c801780cb (127.7.43.65). New cstate: current_term: 1 leader_uuid: "a255e41a83d6499ea839047c801780cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40a5a24cba3b489885e1df5b3febda4d" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a6fc9246e5d40f3a293032c65a7f6a2" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 35147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a255e41a83d6499ea839047c801780cb" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 44971 } health_report { overall_health: HEALTHY } } }
I20260806 06:31:08.436976 9609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:08.434775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39240 (local address 127.7.43.65:44971)
0806 06:31:08.434813 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:08.434815 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:08.434856 (+ 41us) server_negotiation.cc:407] Connection header received
0806 06:31:08.434920 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:08.434920 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:08.434935 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:08.434986 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:31:08.435078 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.435493 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.436177 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.436344 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.436383 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:08.436511 (+ 128us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:31:08.436588 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:31:08.436637 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:08.436663 (+ 26us) server_negotiation.cc:299] Negotiation successful
0806 06:31:08.436677 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:31:08.438514 9318 raft_consensus.cc:1261] T 390e5b5258874d5ea7b828880607edfa P 40a5a24cba3b489885e1df5b3febda4d [term 1 FOLLOWER]: Refusing update from remote peer a255e41a83d6499ea839047c801780cb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:31:08.438870 9630 consensus_queue.cc:1035] T 390e5b5258874d5ea7b828880607edfa P a255e41a83d6499ea839047c801780cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "40a5a24cba3b489885e1df5b3febda4d" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:31:08.441507 9515 raft_consensus.cc:1261] T 390e5b5258874d5ea7b828880607edfa P 6a6fc9246e5d40f3a293032c65a7f6a2 [term 1 FOLLOWER]: Refusing update from remote peer a255e41a83d6499ea839047c801780cb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:31:08.441771 9626 consensus_queue.cc:1035] T 390e5b5258874d5ea7b828880607edfa P a255e41a83d6499ea839047c801780cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a6fc9246e5d40f3a293032c65a7f6a2" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 35147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:31:08.442699 9634 mvcc.cc:205] Tried to move back new op lower bound from 7315447269121626112 to 7315447269021630464. Current Snapshot: MvccSnapshot[applied={T|T < 7315447269121626112}]
I20260806 06:31:08.446189 8894 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47500:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:31:08.446326 8894 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47500:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:31:08.447203 8894 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1b10352f33a04b8fa06d09a309070f3b: Sending DeleteTablet for 3 replicas of tablet 390e5b5258874d5ea7b828880607edfa
I20260806 06:31:08.447618 9289 tablet_service.cc:1514] Processing DeleteTablet for tablet 390e5b5258874d5ea7b828880607edfa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:31:08 UTC) from {username='kudu', principal='kudu/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:56956
I20260806 06:31:08.447623 9486 tablet_service.cc:1514] Processing DeleteTablet for tablet 390e5b5258874d5ea7b828880607edfa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:31:08 UTC) from {username='kudu', principal='kudu/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:48436
I20260806 06:31:08.448035 9650 tablet_replica.cc:330] T 390e5b5258874d5ea7b828880607edfa P 6a6fc9246e5d40f3a293032c65a7f6a2: stopping tablet replica
I20260806 06:31:08.448117 9650 raft_consensus.cc:2229] T 390e5b5258874d5ea7b828880607edfa P 6a6fc9246e5d40f3a293032c65a7f6a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:31:08.448191 9650 raft_consensus.cc:2258] T 390e5b5258874d5ea7b828880607edfa P 6a6fc9246e5d40f3a293032c65a7f6a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:31:08.448501 9650 ts_tablet_manager.cc:1905] T 390e5b5258874d5ea7b828880607edfa P 6a6fc9246e5d40f3a293032c65a7f6a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:31:08.450102 9650 ts_tablet_manager.cc:1918] T 390e5b5258874d5ea7b828880607edfa P 6a6fc9246e5d40f3a293032c65a7f6a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:31:08.450181 9650 log.cc:1196] T 390e5b5258874d5ea7b828880607edfa P 6a6fc9246e5d40f3a293032c65a7f6a2: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/ts-2/wal/wals/390e5b5258874d5ea7b828880607edfa
I20260806 06:31:08.450170 9096 tablet_service.cc:1514] Processing DeleteTablet for tablet 390e5b5258874d5ea7b828880607edfa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:31:08 UTC) from {username='kudu', principal='kudu/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:39230
I20260806 06:31:08.450383 8894 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47500:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 06:31:08.450454 9650 ts_tablet_manager.cc:1939] T 390e5b5258874d5ea7b828880607edfa P 6a6fc9246e5d40f3a293032c65a7f6a2: Deleting consensus metadata
I20260806 06:31:08.450445 9651 tablet_replica.cc:330] T 390e5b5258874d5ea7b828880607edfa P a255e41a83d6499ea839047c801780cb: stopping tablet replica
W20260806 06:31:08.450525 8894 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:31:08.450722 9651 raft_consensus.cc:2229] T 390e5b5258874d5ea7b828880607edfa P a255e41a83d6499ea839047c801780cb [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:31:08.450881 9651 raft_consensus.cc:2258] T 390e5b5258874d5ea7b828880607edfa P a255e41a83d6499ea839047c801780cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:31:08.451072 9649 tablet_replica.cc:330] T 390e5b5258874d5ea7b828880607edfa P 40a5a24cba3b489885e1df5b3febda4d: stopping tablet replica
I20260806 06:31:08.451181 9649 raft_consensus.cc:2229] T 390e5b5258874d5ea7b828880607edfa P 40a5a24cba3b489885e1df5b3febda4d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:31:08.451299 9649 raft_consensus.cc:2258] T 390e5b5258874d5ea7b828880607edfa P 40a5a24cba3b489885e1df5b3febda4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:31:08.451570 9651 ts_tablet_manager.cc:1905] T 390e5b5258874d5ea7b828880607edfa P a255e41a83d6499ea839047c801780cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:31:08.451706 9649 ts_tablet_manager.cc:1905] T 390e5b5258874d5ea7b828880607edfa P 40a5a24cba3b489885e1df5b3febda4d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:31:08.452028 9486 tablet_service.cc:1467] Processing CreateTablet for tablet 947c3950c2a44b66a228e0e8757a3828 (DEFAULT_TABLE table=test-table [id=48bea342222e4334a93e40f61161816f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:31:08.452142 9486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 947c3950c2a44b66a228e0e8757a3828. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:08.452126 9096 tablet_service.cc:1467] Processing CreateTablet for tablet 947c3950c2a44b66a228e0e8757a3828 (DEFAULT_TABLE table=test-table [id=48bea342222e4334a93e40f61161816f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:31:08.452224 9096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 947c3950c2a44b66a228e0e8757a3828. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:08.453608 9613 tablet_bootstrap.cc:492] T 947c3950c2a44b66a228e0e8757a3828 P 6a6fc9246e5d40f3a293032c65a7f6a2: Bootstrap starting.
I20260806 06:31:08.453616 9289 tablet_service.cc:1467] Processing CreateTablet for tablet 947c3950c2a44b66a228e0e8757a3828 (DEFAULT_TABLE table=test-table [id=48bea342222e4334a93e40f61161816f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:31:08.453845 9651 ts_tablet_manager.cc:1918] T 390e5b5258874d5ea7b828880607edfa P a255e41a83d6499ea839047c801780cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:31:08.453966 9651 log.cc:1196] T 390e5b5258874d5ea7b828880607edfa P a255e41a83d6499ea839047c801780cb: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/ts-0/wal/wals/390e5b5258874d5ea7b828880607edfa
I20260806 06:31:08.454108 9613 tablet_bootstrap.cc:654] T 947c3950c2a44b66a228e0e8757a3828 P 6a6fc9246e5d40f3a293032c65a7f6a2: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:08.453984 8874 catalog_manager.cc:4841] TS 6a6fc9246e5d40f3a293032c65a7f6a2 (127.7.43.67:35147): tablet 390e5b5258874d5ea7b828880607edfa (table test-table [id=b85f7326a44f45b68f667796936e1cac]) successfully deleted
I20260806 06:31:08.454464 9651 ts_tablet_manager.cc:1939] T 390e5b5258874d5ea7b828880607edfa P a255e41a83d6499ea839047c801780cb: Deleting consensus metadata
I20260806 06:31:08.454830 9614 tablet_bootstrap.cc:492] T 947c3950c2a44b66a228e0e8757a3828 P a255e41a83d6499ea839047c801780cb: Bootstrap starting.
I20260806 06:31:08.455169 9613 tablet_bootstrap.cc:492] T 947c3950c2a44b66a228e0e8757a3828 P 6a6fc9246e5d40f3a293032c65a7f6a2: No bootstrap required, opened a new log
I20260806 06:31:08.455230 9613 ts_tablet_manager.cc:1397] T 947c3950c2a44b66a228e0e8757a3828 P 6a6fc9246e5d40f3a293032c65a7f6a2: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 06:31:08.455351 9614 tablet_bootstrap.cc:654] T 947c3950c2a44b66a228e0e8757a3828 P a255e41a83d6499ea839047c801780cb: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:08.455549 9613 raft_consensus.cc:348] T 947c3950c2a44b66a228e0e8757a3828 P 6a6fc9246e5d40f3a293032c65a7f6a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a255e41a83d6499ea839047c801780cb" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 44971 } } peers { permanent_uuid: "40a5a24cba3b489885e1df5b3febda4d" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37701 } } peers { permanent_uuid: "6a6fc9246e5d40f3a293032c65a7f6a2" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 35147 } }
I20260806 06:31:08.455632 9613 raft_consensus.cc:374] T 947c3950c2a44b66a228e0e8757a3828 P 6a6fc9246e5d40f3a293032c65a7f6a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:08.455654 9613 raft_consensus.cc:729] T 947c3950c2a44b66a228e0e8757a3828 P 6a6fc9246e5d40f3a293032c65a7f6a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a6fc9246e5d40f3a293032c65a7f6a2, State: Initialized, Role: FOLLOWER
I20260806 06:31:08.455704 9613 consensus_queue.cc:260] T 947c3950c2a44b66a228e0e8757a3828 P 6a6fc9246e5d40f3a293032c65a7f6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a255e41a83d6499ea839047c801780cb" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 44971 } } peers { permanent_uuid: "40a5a24cba3b489885e1df5b3febda4d" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37701 } } peers { permanent_uuid: "6a6fc9246e5d40f3a293032c65a7f6a2" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 35147 } }
I20260806 06:31:08.455790 9613 ts_tablet_manager.cc:1428] T 947c3950c2a44b66a228e0e8757a3828 P 6a6fc9246e5d40f3a293032c65a7f6a2: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 06:31:08.456056 8876 catalog_manager.cc:4841] TS a255e41a83d6499ea839047c801780cb (127.7.43.65:44971): tablet 390e5b5258874d5ea7b828880607edfa (table test-table [id=b85f7326a44f45b68f667796936e1cac]) successfully deleted
I20260806 06:31:08.456195 9289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 947c3950c2a44b66a228e0e8757a3828. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:08.457357 9620 tablet_bootstrap.cc:492] T 947c3950c2a44b66a228e0e8757a3828 P 40a5a24cba3b489885e1df5b3febda4d: Bootstrap starting.
I20260806 06:31:08.458209 9620 tablet_bootstrap.cc:654] T 947c3950c2a44b66a228e0e8757a3828 P 40a5a24cba3b489885e1df5b3febda4d: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:08.458664 9614 tablet_bootstrap.cc:492] T 947c3950c2a44b66a228e0e8757a3828 P a255e41a83d6499ea839047c801780cb: No bootstrap required, opened a new log
I20260806 06:31:08.458709 9614 ts_tablet_manager.cc:1397] T 947c3950c2a44b66a228e0e8757a3828 P a255e41a83d6499ea839047c801780cb: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260806 06:31:08.459017 9614 raft_consensus.cc:348] T 947c3950c2a44b66a228e0e8757a3828 P a255e41a83d6499ea839047c801780cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a255e41a83d6499ea839047c801780cb" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 44971 } } peers { permanent_uuid: "40a5a24cba3b489885e1df5b3febda4d" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37701 } } peers { permanent_uuid: "6a6fc9246e5d40f3a293032c65a7f6a2" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 35147 } }
I20260806 06:31:08.459110 9614 raft_consensus.cc:374] T 947c3950c2a44b66a228e0e8757a3828 P a255e41a83d6499ea839047c801780cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:08.459137 9614 raft_consensus.cc:729] T 947c3950c2a44b66a228e0e8757a3828 P a255e41a83d6499ea839047c801780cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a255e41a83d6499ea839047c801780cb, State: Initialized, Role: FOLLOWER
I20260806 06:31:08.459180 9620 tablet_bootstrap.cc:492] T 947c3950c2a44b66a228e0e8757a3828 P 40a5a24cba3b489885e1df5b3febda4d: No bootstrap required, opened a new log
I20260806 06:31:08.459172 9614 consensus_queue.cc:260] T 947c3950c2a44b66a228e0e8757a3828 P a255e41a83d6499ea839047c801780cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a255e41a83d6499ea839047c801780cb" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 44971 } } peers { permanent_uuid: "40a5a24cba3b489885e1df5b3febda4d" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37701 } } peers { permanent_uuid: "6a6fc9246e5d40f3a293032c65a7f6a2" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 35147 } }
I20260806 06:31:08.459232 9620 ts_tablet_manager.cc:1397] T 947c3950c2a44b66a228e0e8757a3828 P 40a5a24cba3b489885e1df5b3febda4d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 06:31:08.459256 9614 ts_tablet_manager.cc:1428] T 947c3950c2a44b66a228e0e8757a3828 P a255e41a83d6499ea839047c801780cb: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 06:31:08.459472 9620 raft_consensus.cc:348] T 947c3950c2a44b66a228e0e8757a3828 P 40a5a24cba3b489885e1df5b3febda4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a255e41a83d6499ea839047c801780cb" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 44971 } } peers { permanent_uuid: "40a5a24cba3b489885e1df5b3febda4d" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37701 } } peers { permanent_uuid: "6a6fc9246e5d40f3a293032c65a7f6a2" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 35147 } }
I20260806 06:31:08.459545 9620 raft_consensus.cc:374] T 947c3950c2a44b66a228e0e8757a3828 P 40a5a24cba3b489885e1df5b3febda4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:08.459565 9620 raft_consensus.cc:729] T 947c3950c2a44b66a228e0e8757a3828 P 40a5a24cba3b489885e1df5b3febda4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40a5a24cba3b489885e1df5b3febda4d, State: Initialized, Role: FOLLOWER
I20260806 06:31:08.459585 9649 ts_tablet_manager.cc:1918] T 390e5b5258874d5ea7b828880607edfa P 40a5a24cba3b489885e1df5b3febda4d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:31:08.459607 9620 consensus_queue.cc:260] T 947c3950c2a44b66a228e0e8757a3828 P 40a5a24cba3b489885e1df5b3febda4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a255e41a83d6499ea839047c801780cb" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 44971 } } peers { permanent_uuid: "40a5a24cba3b489885e1df5b3febda4d" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37701 } } peers { permanent_uuid: "6a6fc9246e5d40f3a293032c65a7f6a2" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 35147 } }
I20260806 06:31:08.459649 9649 log.cc:1196] T 390e5b5258874d5ea7b828880607edfa P 40a5a24cba3b489885e1df5b3febda4d: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/ts-1/wal/wals/390e5b5258874d5ea7b828880607edfa
I20260806 06:31:08.459688 9620 ts_tablet_manager.cc:1428] T 947c3950c2a44b66a228e0e8757a3828 P 40a5a24cba3b489885e1df5b3febda4d: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 06:31:08.459870 9649 ts_tablet_manager.cc:1939] T 390e5b5258874d5ea7b828880607edfa P 40a5a24cba3b489885e1df5b3febda4d: Deleting consensus metadata
I20260806 06:31:08.460614 8877 catalog_manager.cc:4841] TS 40a5a24cba3b489885e1df5b3febda4d (127.7.43.66:37701): tablet 390e5b5258874d5ea7b828880607edfa (table test-table [id=b85f7326a44f45b68f667796936e1cac]) successfully deleted
W20260806 06:31:08.499039 9189 tablet.cc:2369] T 947c3950c2a44b66a228e0e8757a3828 P a255e41a83d6499ea839047c801780cb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 06:31:08.558097 9586 tablet.cc:2369] T 947c3950c2a44b66a228e0e8757a3828 P 6a6fc9246e5d40f3a293032c65a7f6a2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:31:08.603497 9622 raft_consensus.cc:482] T 947c3950c2a44b66a228e0e8757a3828 P 6a6fc9246e5d40f3a293032c65a7f6a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:31:08.603608 9622 raft_consensus.cc:504] T 947c3950c2a44b66a228e0e8757a3828 P 6a6fc9246e5d40f3a293032c65a7f6a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a255e41a83d6499ea839047c801780cb" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 44971 } } peers { permanent_uuid: "40a5a24cba3b489885e1df5b3febda4d" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37701 } } peers { permanent_uuid: "6a6fc9246e5d40f3a293032c65a7f6a2" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 35147 } }
I20260806 06:31:08.604110 9622 leader_election.cc:288] T 947c3950c2a44b66a228e0e8757a3828 P 6a6fc9246e5d40f3a293032c65a7f6a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a255e41a83d6499ea839047c801780cb (127.7.43.65:44971), 40a5a24cba3b489885e1df5b3febda4d (127.7.43.66:37701)
I20260806 06:31:08.608042 9608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:08.604415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:41175 (local address 127.7.43.66:37701)
0806 06:31:08.604473 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:08.604475 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:08.604680 (+ 205us) server_negotiation.cc:407] Connection header received
0806 06:31:08.604713 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:08.604714 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:08.604723 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:08.604796 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:31:08.605007 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.605722 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.607167 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.607656 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.607723 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:08.607735 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:08.607916 (+ 181us) server_negotiation.cc:299] Negotiation successful
0806 06:31:08.607969 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 06:31:08.608054 9657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:08.604432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.66:37701 (local address 127.7.43.67:41175)
0806 06:31:08.604587 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:08.604595 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:08.604658 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:08.604850 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:08.604851 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:08.604853 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:31:08.604944 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:08.604946 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.605785 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.605787 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:08.607084 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:08.607087 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.607785 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.607787 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:08.607853 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:08.607857 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:31:08.607908 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 06:31:08.607946 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":1,"thread_start_us":86,"threads_started":1}
I20260806 06:31:08.608315 9609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:08.604522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:60771 (local address 127.7.43.65:44971)
0806 06:31:08.604568 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:08.604570 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:08.604578 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:31:08.604610 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:08.604611 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:08.604620 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:08.604662 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:31:08.604912 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.605656 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.607126 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.607753 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.607823 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:08.607833 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:08.608164 (+ 331us) server_negotiation.cc:299] Negotiation successful
0806 06:31:08.608251 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 06:31:08.608541 9427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:08.604295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:44971 (local address 127.7.43.67:60771)
0806 06:31:08.604377 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:08.604391 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:08.604440 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:08.604712 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:08.604713 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:08.604714 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:31:08.604830 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:08.604833 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.605750 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.605752 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:08.607026 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:08.607029 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.607824 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.607825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:08.607972 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:08.607976 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:31:08.608088 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 06:31:08.608479 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260806 06:31:08.608741 9318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "947c3950c2a44b66a228e0e8757a3828" candidate_uuid: "6a6fc9246e5d40f3a293032c65a7f6a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40a5a24cba3b489885e1df5b3febda4d" is_pre_election: true
I20260806 06:31:08.608768 9120 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "947c3950c2a44b66a228e0e8757a3828" candidate_uuid: "6a6fc9246e5d40f3a293032c65a7f6a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a255e41a83d6499ea839047c801780cb" is_pre_election: true
I20260806 06:31:08.608848 9318 raft_consensus.cc:2454] T 947c3950c2a44b66a228e0e8757a3828 P 40a5a24cba3b489885e1df5b3febda4d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a6fc9246e5d40f3a293032c65a7f6a2 in term 0.
I20260806 06:31:08.608902 9120 raft_consensus.cc:2454] T 947c3950c2a44b66a228e0e8757a3828 P a255e41a83d6499ea839047c801780cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a6fc9246e5d40f3a293032c65a7f6a2 in term 0.
I20260806 06:31:08.609076 9422 leader_election.cc:302] T 947c3950c2a44b66a228e0e8757a3828 P 6a6fc9246e5d40f3a293032c65a7f6a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40a5a24cba3b489885e1df5b3febda4d, 6a6fc9246e5d40f3a293032c65a7f6a2; no voters:
I20260806 06:31:08.609213 9622 raft_consensus.cc:2781] T 947c3950c2a44b66a228e0e8757a3828 P 6a6fc9246e5d40f3a293032c65a7f6a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:31:08.609280 9622 raft_consensus.cc:482] T 947c3950c2a44b66a228e0e8757a3828 P 6a6fc9246e5d40f3a293032c65a7f6a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:31:08.609315 9622 raft_consensus.cc:3037] T 947c3950c2a44b66a228e0e8757a3828 P 6a6fc9246e5d40f3a293032c65a7f6a2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:08.610508 9622 raft_consensus.cc:504] T 947c3950c2a44b66a228e0e8757a3828 P 6a6fc9246e5d40f3a293032c65a7f6a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a255e41a83d6499ea839047c801780cb" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 44971 } } peers { permanent_uuid: "40a5a24cba3b489885e1df5b3febda4d" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37701 } } peers { permanent_uuid: "6a6fc9246e5d40f3a293032c65a7f6a2" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 35147 } }
I20260806 06:31:08.610622 9622 leader_election.cc:288] T 947c3950c2a44b66a228e0e8757a3828 P 6a6fc9246e5d40f3a293032c65a7f6a2 [CANDIDATE]: Term 1 election: Requested vote from peers a255e41a83d6499ea839047c801780cb (127.7.43.65:44971), 40a5a24cba3b489885e1df5b3febda4d (127.7.43.66:37701)
I20260806 06:31:08.610762 9120 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "947c3950c2a44b66a228e0e8757a3828" candidate_uuid: "6a6fc9246e5d40f3a293032c65a7f6a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a255e41a83d6499ea839047c801780cb"
I20260806 06:31:08.610796 9318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "947c3950c2a44b66a228e0e8757a3828" candidate_uuid: "6a6fc9246e5d40f3a293032c65a7f6a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40a5a24cba3b489885e1df5b3febda4d"
I20260806 06:31:08.610841 9120 raft_consensus.cc:3037] T 947c3950c2a44b66a228e0e8757a3828 P a255e41a83d6499ea839047c801780cb [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:08.610857 9318 raft_consensus.cc:3037] T 947c3950c2a44b66a228e0e8757a3828 P 40a5a24cba3b489885e1df5b3febda4d [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:08.611660 9120 raft_consensus.cc:2454] T 947c3950c2a44b66a228e0e8757a3828 P a255e41a83d6499ea839047c801780cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a6fc9246e5d40f3a293032c65a7f6a2 in term 1.
I20260806 06:31:08.611755 9318 raft_consensus.cc:2454] T 947c3950c2a44b66a228e0e8757a3828 P 40a5a24cba3b489885e1df5b3febda4d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a6fc9246e5d40f3a293032c65a7f6a2 in term 1.
I20260806 06:31:08.611858 9420 leader_election.cc:302] T 947c3950c2a44b66a228e0e8757a3828 P 6a6fc9246e5d40f3a293032c65a7f6a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a6fc9246e5d40f3a293032c65a7f6a2, a255e41a83d6499ea839047c801780cb; no voters:
I20260806 06:31:08.611966 9622 raft_consensus.cc:2781] T 947c3950c2a44b66a228e0e8757a3828 P 6a6fc9246e5d40f3a293032c65a7f6a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:31:08.612133 9622 raft_consensus.cc:686] T 947c3950c2a44b66a228e0e8757a3828 P 6a6fc9246e5d40f3a293032c65a7f6a2 [term 1 LEADER]: Becoming Leader. State: Replica: 6a6fc9246e5d40f3a293032c65a7f6a2, State: Running, Role: LEADER
I20260806 06:31:08.612200 9622 consensus_queue.cc:237] T 947c3950c2a44b66a228e0e8757a3828 P 6a6fc9246e5d40f3a293032c65a7f6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a255e41a83d6499ea839047c801780cb" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 44971 } } peers { permanent_uuid: "40a5a24cba3b489885e1df5b3febda4d" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37701 } } peers { permanent_uuid: "6a6fc9246e5d40f3a293032c65a7f6a2" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 35147 } }
I20260806 06:31:08.612857 8894 catalog_manager.cc:5495] T 947c3950c2a44b66a228e0e8757a3828 P 6a6fc9246e5d40f3a293032c65a7f6a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a6fc9246e5d40f3a293032c65a7f6a2 (127.7.43.67). New cstate: current_term: 1 leader_uuid: "6a6fc9246e5d40f3a293032c65a7f6a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a255e41a83d6499ea839047c801780cb" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 44971 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40a5a24cba3b489885e1df5b3febda4d" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a6fc9246e5d40f3a293032c65a7f6a2" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 35147 } health_report { overall_health: HEALTHY } } }
Aug 06 06:31:08 dist-test-slave-6zbq krb5kdc[8837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997867, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.126@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:31:08.669198 9666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:08.659012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:36741 (local address 127.0.0.1:47502)
0806 06:31:08.659193 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:08.659203 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:08.659278 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:08.659396 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:08.659397 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:08.659715 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:08.659763 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:08.659766 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.660391 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.660392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:08.660901 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:08.660903 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.661287 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.661288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:08.661355 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:08.665820 (+ 4465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:08.665824 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:08.667176 (+ 1352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:08.668377 (+ 1201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:08.668383 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:08.668384 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:08.668565 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:08.668715 (+ 150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:08.668715 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:08.668715 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:08.668767 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:08.668934 (+ 167us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:08.668938 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:08.668977 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 06:31:08.669021 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 06:31:08.669106 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":83,"threads_started":1}
I20260806 06:31:08.669255 8968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:08.659063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47502 (local address 127.7.43.126:36741)
0806 06:31:08.659118 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:08.659120 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:08.659268 (+ 148us) server_negotiation.cc:407] Connection header received
0806 06:31:08.659330 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:08.659330 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:08.659341 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:08.659388 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:08.659846 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.660322 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.660978 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.661202 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.661264 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:08.667266 (+ 6002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:08.667268 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:08.667268 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:08.667275 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:08.668313 (+ 1038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:08.668619 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:08.668619 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:08.668619 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:08.668671 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:08.668805 (+ 134us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:08.668806 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:08.668806 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:08.668895 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:08.668938 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:08.669044 (+ 106us) server_negotiation.cc:299] Negotiation successful
0806 06:31:08.669147 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":29}
I20260806 06:31:08.671216 8894 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:47502:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 06:31:08.671463 8894 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:31:08.672755 9096 tablet_service.cc:1467] Processing CreateTablet for tablet d05be15b8f99465398895fda2edba215 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5fd02f368451420ba6157225bd64cb0e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 06:31:08.672768 9289 tablet_service.cc:1467] Processing CreateTablet for tablet d05be15b8f99465398895fda2edba215 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5fd02f368451420ba6157225bd64cb0e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 06:31:08.672776 9486 tablet_service.cc:1467] Processing CreateTablet for tablet d05be15b8f99465398895fda2edba215 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5fd02f368451420ba6157225bd64cb0e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 06:31:08.672957 9096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d05be15b8f99465398895fda2edba215. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:08.672959 9289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d05be15b8f99465398895fda2edba215. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:08.672958 9486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d05be15b8f99465398895fda2edba215. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:08.675139 9614 tablet_bootstrap.cc:492] T d05be15b8f99465398895fda2edba215 P a255e41a83d6499ea839047c801780cb: Bootstrap starting.
I20260806 06:31:08.675177 9620 tablet_bootstrap.cc:492] T d05be15b8f99465398895fda2edba215 P 40a5a24cba3b489885e1df5b3febda4d: Bootstrap starting.
I20260806 06:31:08.675179 9613 tablet_bootstrap.cc:492] T d05be15b8f99465398895fda2edba215 P 6a6fc9246e5d40f3a293032c65a7f6a2: Bootstrap starting.
I20260806 06:31:08.675451 9614 tablet_bootstrap.cc:654] T d05be15b8f99465398895fda2edba215 P a255e41a83d6499ea839047c801780cb: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:08.675575 9613 tablet_bootstrap.cc:654] T d05be15b8f99465398895fda2edba215 P 6a6fc9246e5d40f3a293032c65a7f6a2: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:08.675575 9620 tablet_bootstrap.cc:654] T d05be15b8f99465398895fda2edba215 P 40a5a24cba3b489885e1df5b3febda4d: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:08.676028 9614 tablet_bootstrap.cc:492] T d05be15b8f99465398895fda2edba215 P a255e41a83d6499ea839047c801780cb: No bootstrap required, opened a new log
I20260806 06:31:08.676081 9614 ts_tablet_manager.cc:1397] T d05be15b8f99465398895fda2edba215 P a255e41a83d6499ea839047c801780cb: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260806 06:31:08.676177 9620 tablet_bootstrap.cc:492] T d05be15b8f99465398895fda2edba215 P 40a5a24cba3b489885e1df5b3febda4d: No bootstrap required, opened a new log
I20260806 06:31:08.676221 9620 ts_tablet_manager.cc:1397] T d05be15b8f99465398895fda2edba215 P 40a5a24cba3b489885e1df5b3febda4d: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 06:31:08.676199 9614 raft_consensus.cc:348] T d05be15b8f99465398895fda2edba215 P a255e41a83d6499ea839047c801780cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a255e41a83d6499ea839047c801780cb" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 44971 } } peers { permanent_uuid: "40a5a24cba3b489885e1df5b3febda4d" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37701 } } peers { permanent_uuid: "6a6fc9246e5d40f3a293032c65a7f6a2" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 35147 } }
I20260806 06:31:08.676268 9614 raft_consensus.cc:374] T d05be15b8f99465398895fda2edba215 P a255e41a83d6499ea839047c801780cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:08.676293 9614 raft_consensus.cc:729] T d05be15b8f99465398895fda2edba215 P a255e41a83d6499ea839047c801780cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a255e41a83d6499ea839047c801780cb, State: Initialized, Role: FOLLOWER
I20260806 06:31:08.676337 9614 consensus_queue.cc:260] T d05be15b8f99465398895fda2edba215 P a255e41a83d6499ea839047c801780cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a255e41a83d6499ea839047c801780cb" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 44971 } } peers { permanent_uuid: "40a5a24cba3b489885e1df5b3febda4d" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37701 } } peers { permanent_uuid: "6a6fc9246e5d40f3a293032c65a7f6a2" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 35147 } }
I20260806 06:31:08.676375 9620 raft_consensus.cc:348] T d05be15b8f99465398895fda2edba215 P 40a5a24cba3b489885e1df5b3febda4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a255e41a83d6499ea839047c801780cb" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 44971 } } peers { permanent_uuid: "40a5a24cba3b489885e1df5b3febda4d" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37701 } } peers { permanent_uuid: "6a6fc9246e5d40f3a293032c65a7f6a2" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 35147 } }
I20260806 06:31:08.676426 9614 ts_tablet_manager.cc:1428] T d05be15b8f99465398895fda2edba215 P a255e41a83d6499ea839047c801780cb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:08.676442 9620 raft_consensus.cc:374] T d05be15b8f99465398895fda2edba215 P 40a5a24cba3b489885e1df5b3febda4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:08.676465 9620 raft_consensus.cc:729] T d05be15b8f99465398895fda2edba215 P 40a5a24cba3b489885e1df5b3febda4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40a5a24cba3b489885e1df5b3febda4d, State: Initialized, Role: FOLLOWER
I20260806 06:31:08.676455 9626 tablet_replica.cc:439] T d05be15b8f99465398895fda2edba215 P a255e41a83d6499ea839047c801780cb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a255e41a83d6499ea839047c801780cb" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 44971 } } peers { permanent_uuid: "40a5a24cba3b489885e1df5b3febda4d" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37701 } } peers { permanent_uuid: "6a6fc9246e5d40f3a293032c65a7f6a2" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 35147 } } }
I20260806 06:31:08.676504 9620 consensus_queue.cc:260] T d05be15b8f99465398895fda2edba215 P 40a5a24cba3b489885e1df5b3febda4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a255e41a83d6499ea839047c801780cb" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 44971 } } peers { permanent_uuid: "40a5a24cba3b489885e1df5b3febda4d" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37701 } } peers { permanent_uuid: "6a6fc9246e5d40f3a293032c65a7f6a2" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 35147 } }
I20260806 06:31:08.676532 9626 tablet_replica.cc:442] T d05be15b8f99465398895fda2edba215 P a255e41a83d6499ea839047c801780cb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:31:08.676579 9620 ts_tablet_manager.cc:1428] T d05be15b8f99465398895fda2edba215 P 40a5a24cba3b489885e1df5b3febda4d: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 06:31:08.676820 9627 tablet_replica.cc:439] T d05be15b8f99465398895fda2edba215 P 40a5a24cba3b489885e1df5b3febda4d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a255e41a83d6499ea839047c801780cb" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 44971 } } peers { permanent_uuid: "40a5a24cba3b489885e1df5b3febda4d" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37701 } } peers { permanent_uuid: "6a6fc9246e5d40f3a293032c65a7f6a2" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 35147 } } }
I20260806 06:31:08.676887 9613 tablet_bootstrap.cc:492] T d05be15b8f99465398895fda2edba215 P 6a6fc9246e5d40f3a293032c65a7f6a2: No bootstrap required, opened a new log
I20260806 06:31:08.676920 9627 tablet_replica.cc:442] T d05be15b8f99465398895fda2edba215 P 40a5a24cba3b489885e1df5b3febda4d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:31:08.676931 9613 ts_tablet_manager.cc:1397] T d05be15b8f99465398895fda2edba215 P 6a6fc9246e5d40f3a293032c65a7f6a2: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 06:31:08.677130 9613 raft_consensus.cc:348] T d05be15b8f99465398895fda2edba215 P 6a6fc9246e5d40f3a293032c65a7f6a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a255e41a83d6499ea839047c801780cb" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 44971 } } peers { permanent_uuid: "40a5a24cba3b489885e1df5b3febda4d" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37701 } } peers { permanent_uuid: "6a6fc9246e5d40f3a293032c65a7f6a2" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 35147 } }
I20260806 06:31:08.677237 9613 raft_consensus.cc:374] T d05be15b8f99465398895fda2edba215 P 6a6fc9246e5d40f3a293032c65a7f6a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:08.677268 9613 raft_consensus.cc:729] T d05be15b8f99465398895fda2edba215 P 6a6fc9246e5d40f3a293032c65a7f6a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a6fc9246e5d40f3a293032c65a7f6a2, State: Initialized, Role: FOLLOWER
I20260806 06:31:08.677314 9613 consensus_queue.cc:260] T d05be15b8f99465398895fda2edba215 P 6a6fc9246e5d40f3a293032c65a7f6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a255e41a83d6499ea839047c801780cb" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 44971 } } peers { permanent_uuid: "40a5a24cba3b489885e1df5b3febda4d" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37701 } } peers { permanent_uuid: "6a6fc9246e5d40f3a293032c65a7f6a2" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 35147 } }
I20260806 06:31:08.677408 9613 ts_tablet_manager.cc:1428] T d05be15b8f99465398895fda2edba215 P 6a6fc9246e5d40f3a293032c65a7f6a2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:08.677525 9622 tablet_replica.cc:439] T d05be15b8f99465398895fda2edba215 P 6a6fc9246e5d40f3a293032c65a7f6a2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a255e41a83d6499ea839047c801780cb" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 44971 } } peers { permanent_uuid: "40a5a24cba3b489885e1df5b3febda4d" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37701 } } peers { permanent_uuid: "6a6fc9246e5d40f3a293032c65a7f6a2" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 35147 } } }
I20260806 06:31:08.677603 9622 tablet_replica.cc:442] T d05be15b8f99465398895fda2edba215 P 6a6fc9246e5d40f3a293032c65a7f6a2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:31:08.910923 9627 raft_consensus.cc:482] T d05be15b8f99465398895fda2edba215 P 40a5a24cba3b489885e1df5b3febda4d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:31:08.911031 9627 raft_consensus.cc:504] T d05be15b8f99465398895fda2edba215 P 40a5a24cba3b489885e1df5b3febda4d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a255e41a83d6499ea839047c801780cb" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 44971 } } peers { permanent_uuid: "40a5a24cba3b489885e1df5b3febda4d" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37701 } } peers { permanent_uuid: "6a6fc9246e5d40f3a293032c65a7f6a2" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 35147 } }
I20260806 06:31:08.911404 9627 leader_election.cc:288] T d05be15b8f99465398895fda2edba215 P 40a5a24cba3b489885e1df5b3febda4d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a255e41a83d6499ea839047c801780cb (127.7.43.65:44971), 6a6fc9246e5d40f3a293032c65a7f6a2 (127.7.43.67:35147)
I20260806 06:31:08.914170 9676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:08.911642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:44817 (local address 127.7.43.67:35147)
0806 06:31:08.911747 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:08.911749 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:08.911779 (+ 30us) server_negotiation.cc:407] Connection header received
0806 06:31:08.911857 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:08.911858 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:08.911866 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:08.911921 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:31:08.912060 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.912549 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.913462 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.913840 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.913901 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:08.913914 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:08.914047 (+ 133us) server_negotiation.cc:299] Negotiation successful
0806 06:31:08.914091 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":58,"threads_started":1}
I20260806 06:31:08.914155 9672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:08.911569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.67:35147 (local address 127.7.43.66:44817)
0806 06:31:08.911718 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:08.911729 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:08.911788 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:08.911927 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:08.911928 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:08.911930 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:31:08.911999 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:08.912001 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.912628 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.912629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:08.913375 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:08.913377 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.913920 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.913921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:08.913984 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:08.913988 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:31:08.914034 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 06:31:08.914071 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":86,"threads_started":1}
I20260806 06:31:08.914211 9609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:08.911615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:57395 (local address 127.7.43.65:44971)
0806 06:31:08.911660 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:08.911663 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:08.911784 (+ 121us) server_negotiation.cc:407] Connection header received
0806 06:31:08.911848 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:08.911849 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:08.911860 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:08.911908 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:31:08.912034 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.912511 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.913545 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.913899 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.913958 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:08.913969 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:08.914109 (+ 140us) server_negotiation.cc:299] Negotiation successful
0806 06:31:08.914151 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 06:31:08.914270 9674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:08.911567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:44971 (local address 127.7.43.66:57395)
0806 06:31:08.911708 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:08.911734 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:08.911797 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:08.911913 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:08.911913 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:08.911915 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:31:08.911978 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:08.911980 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.912583 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.912584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:08.913367 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:08.913369 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.913973 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.913974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:08.914050 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:08.914053 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:31:08.914098 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 06:31:08.914136 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":85,"threads_started":1}
I20260806 06:31:08.914420 9515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d05be15b8f99465398895fda2edba215" candidate_uuid: "40a5a24cba3b489885e1df5b3febda4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a6fc9246e5d40f3a293032c65a7f6a2" is_pre_election: true
I20260806 06:31:08.914510 9515 raft_consensus.cc:2454] T d05be15b8f99465398895fda2edba215 P 6a6fc9246e5d40f3a293032c65a7f6a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40a5a24cba3b489885e1df5b3febda4d in term 0.
I20260806 06:31:08.914520 9120 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d05be15b8f99465398895fda2edba215" candidate_uuid: "40a5a24cba3b489885e1df5b3febda4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a255e41a83d6499ea839047c801780cb" is_pre_election: true
I20260806 06:31:08.914611 9120 raft_consensus.cc:2454] T d05be15b8f99465398895fda2edba215 P a255e41a83d6499ea839047c801780cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40a5a24cba3b489885e1df5b3febda4d in term 0.
I20260806 06:31:08.914700 9221 leader_election.cc:302] T d05be15b8f99465398895fda2edba215 P 40a5a24cba3b489885e1df5b3febda4d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40a5a24cba3b489885e1df5b3febda4d, 6a6fc9246e5d40f3a293032c65a7f6a2; no voters:
I20260806 06:31:08.914860 9627 raft_consensus.cc:2781] T d05be15b8f99465398895fda2edba215 P 40a5a24cba3b489885e1df5b3febda4d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:31:08.914891 9627 raft_consensus.cc:482] T d05be15b8f99465398895fda2edba215 P 40a5a24cba3b489885e1df5b3febda4d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:31:08.914911 9627 raft_consensus.cc:3037] T d05be15b8f99465398895fda2edba215 P 40a5a24cba3b489885e1df5b3febda4d [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:08.915568 9627 raft_consensus.cc:504] T d05be15b8f99465398895fda2edba215 P 40a5a24cba3b489885e1df5b3febda4d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a255e41a83d6499ea839047c801780cb" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 44971 } } peers { permanent_uuid: "40a5a24cba3b489885e1df5b3febda4d" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37701 } } peers { permanent_uuid: "6a6fc9246e5d40f3a293032c65a7f6a2" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 35147 } }
I20260806 06:31:08.915686 9627 leader_election.cc:288] T d05be15b8f99465398895fda2edba215 P 40a5a24cba3b489885e1df5b3febda4d [CANDIDATE]: Term 1 election: Requested vote from peers a255e41a83d6499ea839047c801780cb (127.7.43.65:44971), 6a6fc9246e5d40f3a293032c65a7f6a2 (127.7.43.67:35147)
I20260806 06:31:08.915822 9515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d05be15b8f99465398895fda2edba215" candidate_uuid: "40a5a24cba3b489885e1df5b3febda4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a6fc9246e5d40f3a293032c65a7f6a2"
I20260806 06:31:08.915835 9120 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d05be15b8f99465398895fda2edba215" candidate_uuid: "40a5a24cba3b489885e1df5b3febda4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a255e41a83d6499ea839047c801780cb"
I20260806 06:31:08.915895 9120 raft_consensus.cc:3037] T d05be15b8f99465398895fda2edba215 P a255e41a83d6499ea839047c801780cb [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:08.915894 9515 raft_consensus.cc:3037] T d05be15b8f99465398895fda2edba215 P 6a6fc9246e5d40f3a293032c65a7f6a2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:08.916419 9515 raft_consensus.cc:2454] T d05be15b8f99465398895fda2edba215 P 6a6fc9246e5d40f3a293032c65a7f6a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40a5a24cba3b489885e1df5b3febda4d in term 1.
I20260806 06:31:08.916528 9120 raft_consensus.cc:2454] T d05be15b8f99465398895fda2edba215 P a255e41a83d6499ea839047c801780cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40a5a24cba3b489885e1df5b3febda4d in term 1.
I20260806 06:31:08.916599 9221 leader_election.cc:302] T d05be15b8f99465398895fda2edba215 P 40a5a24cba3b489885e1df5b3febda4d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40a5a24cba3b489885e1df5b3febda4d, 6a6fc9246e5d40f3a293032c65a7f6a2; no voters:
I20260806 06:31:08.916687 9627 raft_consensus.cc:2781] T d05be15b8f99465398895fda2edba215 P 40a5a24cba3b489885e1df5b3febda4d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:31:08.916780 9627 raft_consensus.cc:686] T d05be15b8f99465398895fda2edba215 P 40a5a24cba3b489885e1df5b3febda4d [term 1 LEADER]: Becoming Leader. State: Replica: 40a5a24cba3b489885e1df5b3febda4d, State: Running, Role: LEADER
I20260806 06:31:08.916843 9627 consensus_queue.cc:237] T d05be15b8f99465398895fda2edba215 P 40a5a24cba3b489885e1df5b3febda4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a255e41a83d6499ea839047c801780cb" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 44971 } } peers { permanent_uuid: "40a5a24cba3b489885e1df5b3febda4d" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37701 } } peers { permanent_uuid: "6a6fc9246e5d40f3a293032c65a7f6a2" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 35147 } }
I20260806 06:31:08.917140 9677 tablet_replica.cc:439] T d05be15b8f99465398895fda2edba215 P 40a5a24cba3b489885e1df5b3febda4d: TxnStatusTablet state changed. Reason: New leader 40a5a24cba3b489885e1df5b3febda4d. Latest consensus state: current_term: 1 leader_uuid: "40a5a24cba3b489885e1df5b3febda4d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a255e41a83d6499ea839047c801780cb" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 44971 } } peers { permanent_uuid: "40a5a24cba3b489885e1df5b3febda4d" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37701 } } peers { permanent_uuid: "6a6fc9246e5d40f3a293032c65a7f6a2" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 35147 } } }
I20260806 06:31:08.917234 9677 tablet_replica.cc:442] T d05be15b8f99465398895fda2edba215 P 40a5a24cba3b489885e1df5b3febda4d: This TxnStatusTablet replica's current role is: LEADER
I20260806 06:31:08.917495 9679 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 06:31:08.917523 8894 catalog_manager.cc:5495] T d05be15b8f99465398895fda2edba215 P 40a5a24cba3b489885e1df5b3febda4d reported cstate change: term changed from 0 to 1, leader changed from <none> to 40a5a24cba3b489885e1df5b3febda4d (127.7.43.66). New cstate: current_term: 1 leader_uuid: "40a5a24cba3b489885e1df5b3febda4d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a255e41a83d6499ea839047c801780cb" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 44971 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40a5a24cba3b489885e1df5b3febda4d" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a6fc9246e5d40f3a293032c65a7f6a2" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 35147 } health_report { overall_health: UNKNOWN } } }
I20260806 06:31:08.917608 9679 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 06:31:08.957540 9608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:08.955158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56960 (local address 127.7.43.66:37701)
0806 06:31:08.955221 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:08.955223 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:08.955255 (+ 32us) server_negotiation.cc:407] Connection header received
0806 06:31:08.955349 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:08.955350 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:08.955369 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:08.955426 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:31:08.955540 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.956066 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.956786 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.956992 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.957047 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:08.957212 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:31:08.957311 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:31:08.957373 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:08.957428 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 06:31:08.957446 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 06:31:08.957540 9666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:08.955107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.66:37701 (local address 127.0.0.1:56960)
0806 06:31:08.955199 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:08.955212 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:08.955277 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:08.955435 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:08.955436 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:08.955437 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 06:31:08.955487 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:08.955489 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.956143 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.956144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:08.956710 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:08.956713 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.957051 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.957052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:08.957117 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:08.957132 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 06:31:08.957365 (+ 233us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 06:31:08.957373 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 06:31:08.957423 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 06:31:08.957471 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 06:31:08.958791 9515 raft_consensus.cc:1261] T d05be15b8f99465398895fda2edba215 P 6a6fc9246e5d40f3a293032c65a7f6a2 [term 1 FOLLOWER]: Refusing update from remote peer 40a5a24cba3b489885e1df5b3febda4d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:31:08.958858 9120 raft_consensus.cc:1261] T d05be15b8f99465398895fda2edba215 P a255e41a83d6499ea839047c801780cb [term 1 FOLLOWER]: Refusing update from remote peer 40a5a24cba3b489885e1df5b3febda4d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:31:08.959041 9627 consensus_queue.cc:1035] T d05be15b8f99465398895fda2edba215 P 40a5a24cba3b489885e1df5b3febda4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a6fc9246e5d40f3a293032c65a7f6a2" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 35147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:31:08.959290 9627 consensus_queue.cc:1035] T d05be15b8f99465398895fda2edba215 P 40a5a24cba3b489885e1df5b3febda4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a255e41a83d6499ea839047c801780cb" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 44971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:31:08.959952 9626 tablet_replica.cc:439] T d05be15b8f99465398895fda2edba215 P a255e41a83d6499ea839047c801780cb: TxnStatusTablet state changed. Reason: New leader 40a5a24cba3b489885e1df5b3febda4d. Latest consensus state: current_term: 1 leader_uuid: "40a5a24cba3b489885e1df5b3febda4d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a255e41a83d6499ea839047c801780cb" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 44971 } } peers { permanent_uuid: "40a5a24cba3b489885e1df5b3febda4d" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37701 } } peers { permanent_uuid: "6a6fc9246e5d40f3a293032c65a7f6a2" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 35147 } } }
I20260806 06:31:08.960041 9626 tablet_replica.cc:442] T d05be15b8f99465398895fda2edba215 P a255e41a83d6499ea839047c801780cb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:31:08.959976 9622 tablet_replica.cc:439] T d05be15b8f99465398895fda2edba215 P 6a6fc9246e5d40f3a293032c65a7f6a2: TxnStatusTablet state changed. Reason: New leader 40a5a24cba3b489885e1df5b3febda4d. Latest consensus state: current_term: 1 leader_uuid: "40a5a24cba3b489885e1df5b3febda4d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a255e41a83d6499ea839047c801780cb" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 44971 } } peers { permanent_uuid: "40a5a24cba3b489885e1df5b3febda4d" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37701 } } peers { permanent_uuid: "6a6fc9246e5d40f3a293032c65a7f6a2" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 35147 } } }
I20260806 06:31:08.960067 9622 tablet_replica.cc:442] T d05be15b8f99465398895fda2edba215 P 6a6fc9246e5d40f3a293032c65a7f6a2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:31:08.960563 9627 tablet_replica.cc:439] T d05be15b8f99465398895fda2edba215 P 40a5a24cba3b489885e1df5b3febda4d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "40a5a24cba3b489885e1df5b3febda4d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a255e41a83d6499ea839047c801780cb" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 44971 } } peers { permanent_uuid: "40a5a24cba3b489885e1df5b3febda4d" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37701 } } peers { permanent_uuid: "6a6fc9246e5d40f3a293032c65a7f6a2" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 35147 } } }
I20260806 06:31:08.960649 9627 tablet_replica.cc:442] T d05be15b8f99465398895fda2edba215 P 40a5a24cba3b489885e1df5b3febda4d: This TxnStatusTablet replica's current role is: LEADER
I20260806 06:31:08.961140 9677 tablet_replica.cc:439] T d05be15b8f99465398895fda2edba215 P 40a5a24cba3b489885e1df5b3febda4d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "40a5a24cba3b489885e1df5b3febda4d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a255e41a83d6499ea839047c801780cb" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 44971 } } peers { permanent_uuid: "40a5a24cba3b489885e1df5b3febda4d" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37701 } } peers { permanent_uuid: "6a6fc9246e5d40f3a293032c65a7f6a2" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 35147 } } }
I20260806 06:31:08.961229 9677 tablet_replica.cc:442] T d05be15b8f99465398895fda2edba215 P 40a5a24cba3b489885e1df5b3febda4d: This TxnStatusTablet replica's current role is: LEADER
I20260806 06:31:08.961355 9622 tablet_replica.cc:439] T d05be15b8f99465398895fda2edba215 P 6a6fc9246e5d40f3a293032c65a7f6a2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "40a5a24cba3b489885e1df5b3febda4d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a255e41a83d6499ea839047c801780cb" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 44971 } } peers { permanent_uuid: "40a5a24cba3b489885e1df5b3febda4d" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37701 } } peers { permanent_uuid: "6a6fc9246e5d40f3a293032c65a7f6a2" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 35147 } } }
I20260806 06:31:08.961443 9622 tablet_replica.cc:442] T d05be15b8f99465398895fda2edba215 P 6a6fc9246e5d40f3a293032c65a7f6a2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:31:08.961689 9626 tablet_replica.cc:439] T d05be15b8f99465398895fda2edba215 P a255e41a83d6499ea839047c801780cb: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "40a5a24cba3b489885e1df5b3febda4d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a255e41a83d6499ea839047c801780cb" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 44971 } } peers { permanent_uuid: "40a5a24cba3b489885e1df5b3febda4d" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37701 } } peers { permanent_uuid: "6a6fc9246e5d40f3a293032c65a7f6a2" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 35147 } } }
I20260806 06:31:08.961772 9626 tablet_replica.cc:442] T d05be15b8f99465398895fda2edba215 P a255e41a83d6499ea839047c801780cb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:31:08.962028 9684 mvcc.cc:205] Tried to move back new op lower bound from 7315447271253340160 to 7315447271083843584. Current Snapshot: MvccSnapshot[applied={T|T < 7315447271253340160}]
I20260806 06:31:08.965801 9676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:08.963270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48452 (local address 127.7.43.67:35147)
0806 06:31:08.963305 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:08.963306 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:08.963401 (+ 95us) server_negotiation.cc:407] Connection header received
0806 06:31:08.963479 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:08.963480 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:08.963497 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:08.963554 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:31:08.963680 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.964145 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.965183 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.965363 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.965409 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:08.965548 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:31:08.965628 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:31:08.965672 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:08.965729 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 06:31:08.965745 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 06:31:08.970449 9608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:08.967980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:35401 (local address 127.7.43.66:37701)
0806 06:31:08.968021 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:08.968022 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:08.968088 (+ 66us) server_negotiation.cc:407] Connection header received
0806 06:31:08.968166 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:08.968166 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:08.968173 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:08.968210 (+ 37us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:31:08.968344 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.968763 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.969797 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.970005 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.970063 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:08.970271 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:31:08.970293 (+ 22us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:31:08.970325 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:08.970392 (+ 67us) server_negotiation.cc:299] Negotiation successful
0806 06:31:08.970407 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:31:08.970539 9699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:08.967908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.66:37701 (local address 127.7.43.67:35401)
0806 06:31:08.968033 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:08.968044 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:08.968112 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:08.968224 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:08.968224 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:08.968236 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 06:31:08.968293 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:08.968294 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.968823 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.968824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:08.969730 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:08.969732 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.970069 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.970070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:08.970187 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:08.970202 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 06:31:08.970328 (+ 126us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 06:31:08.970329 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 06:31:08.970383 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 06:31:08.970424 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80,"thread_start_us":62,"threads_started":1}
I20260806 06:31:08.972729 9120 raft_consensus.cc:1261] T 947c3950c2a44b66a228e0e8757a3828 P a255e41a83d6499ea839047c801780cb [term 1 FOLLOWER]: Refusing update from remote peer 6a6fc9246e5d40f3a293032c65a7f6a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:31:08.972798 9318 raft_consensus.cc:1261] T 947c3950c2a44b66a228e0e8757a3828 P 40a5a24cba3b489885e1df5b3febda4d [term 1 FOLLOWER]: Refusing update from remote peer 6a6fc9246e5d40f3a293032c65a7f6a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:31:08.972947 9658 consensus_queue.cc:1035] T 947c3950c2a44b66a228e0e8757a3828 P 6a6fc9246e5d40f3a293032c65a7f6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a255e41a83d6499ea839047c801780cb" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 44971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:31:08.973146 9658 consensus_queue.cc:1035] T 947c3950c2a44b66a228e0e8757a3828 P 6a6fc9246e5d40f3a293032c65a7f6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40a5a24cba3b489885e1df5b3febda4d" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:31:08.981266 9705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:08.978989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.67:35147 (local address 127.7.43.66:36525)
0806 06:31:08.979134 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:08.979143 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:08.979193 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:08.979307 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:08.979308 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:08.979309 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 06:31:08.979345 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:08.979347 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.979880 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.979881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:08.980465 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:08.980468 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.980895 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.980896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:08.980953 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:08.980967 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 06:31:08.981144 (+ 177us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 06:31:08.981145 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 06:31:08.981174 (+ 29us) client_negotiation.cc:240] Negotiation successful
0806 06:31:08.981205 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":80,"threads_started":1}
I20260806 06:31:08.981261 9676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:08.979191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:36525 (local address 127.7.43.67:35147)
0806 06:31:08.979227 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:08.979228 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:08.979235 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:31:08.979254 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:08.979255 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:08.979263 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:08.979305 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:31:08.979389 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.979819 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.980539 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:08.980812 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:08.980891 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:08.981039 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:31:08.981071 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:31:08.981144 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:08.981180 (+ 36us) server_negotiation.cc:299] Negotiation successful
0806 06:31:08.981202 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:31:08.990881 8895 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47500:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:31:08.990969 8895 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47500:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:31:08.991542 8895 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1b10352f33a04b8fa06d09a309070f3b: Sending DeleteTablet for 3 replicas of tablet 947c3950c2a44b66a228e0e8757a3828
I20260806 06:31:08.991799 9096 tablet_service.cc:1514] Processing DeleteTablet for tablet 947c3950c2a44b66a228e0e8757a3828 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:31:08 UTC) from {username='kudu', principal='kudu/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:39230
I20260806 06:31:08.991860 9289 tablet_service.cc:1514] Processing DeleteTablet for tablet 947c3950c2a44b66a228e0e8757a3828 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:31:08 UTC) from {username='kudu', principal='kudu/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:56956
I20260806 06:31:08.992014 9486 tablet_service.cc:1514] Processing DeleteTablet for tablet 947c3950c2a44b66a228e0e8757a3828 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:31:08 UTC) from {username='kudu', principal='kudu/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:48436
I20260806 06:31:08.992231 9714 tablet_replica.cc:330] T 947c3950c2a44b66a228e0e8757a3828 P 6a6fc9246e5d40f3a293032c65a7f6a2: stopping tablet replica
I20260806 06:31:08.992306 9714 raft_consensus.cc:2229] T 947c3950c2a44b66a228e0e8757a3828 P 6a6fc9246e5d40f3a293032c65a7f6a2 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:31:08.992381 9714 raft_consensus.cc:2258] T 947c3950c2a44b66a228e0e8757a3828 P 6a6fc9246e5d40f3a293032c65a7f6a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:31:08.993126 9714 ts_tablet_manager.cc:1905] T 947c3950c2a44b66a228e0e8757a3828 P 6a6fc9246e5d40f3a293032c65a7f6a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:31:08.994457 9713 tablet_replica.cc:330] T 947c3950c2a44b66a228e0e8757a3828 P 40a5a24cba3b489885e1df5b3febda4d: stopping tablet replica
I20260806 06:31:08.994525 9713 raft_consensus.cc:2229] T 947c3950c2a44b66a228e0e8757a3828 P 40a5a24cba3b489885e1df5b3febda4d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:31:08.994565 9713 raft_consensus.cc:2258] T 947c3950c2a44b66a228e0e8757a3828 P 40a5a24cba3b489885e1df5b3febda4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:31:08.994784 9713 ts_tablet_manager.cc:1905] T 947c3950c2a44b66a228e0e8757a3828 P 40a5a24cba3b489885e1df5b3febda4d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:31:08.995180 9714 ts_tablet_manager.cc:1918] T 947c3950c2a44b66a228e0e8757a3828 P 6a6fc9246e5d40f3a293032c65a7f6a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:31:08.995286 9714 log.cc:1196] T 947c3950c2a44b66a228e0e8757a3828 P 6a6fc9246e5d40f3a293032c65a7f6a2: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/ts-2/wal/wals/947c3950c2a44b66a228e0e8757a3828
I20260806 06:31:08.995560 9714 ts_tablet_manager.cc:1939] T 947c3950c2a44b66a228e0e8757a3828 P 6a6fc9246e5d40f3a293032c65a7f6a2: Deleting consensus metadata
I20260806 06:31:08.995901 8874 catalog_manager.cc:4841] TS 6a6fc9246e5d40f3a293032c65a7f6a2 (127.7.43.67:35147): tablet 947c3950c2a44b66a228e0e8757a3828 (table test-table [id=48bea342222e4334a93e40f61161816f]) successfully deleted
I20260806 06:31:08.996085 9712 tablet_replica.cc:330] T 947c3950c2a44b66a228e0e8757a3828 P a255e41a83d6499ea839047c801780cb: stopping tablet replica
I20260806 06:31:08.996184 9712 raft_consensus.cc:2229] T 947c3950c2a44b66a228e0e8757a3828 P a255e41a83d6499ea839047c801780cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:31:08.996269 9712 raft_consensus.cc:2258] T 947c3950c2a44b66a228e0e8757a3828 P a255e41a83d6499ea839047c801780cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:31:08.996534 9712 ts_tablet_manager.cc:1905] T 947c3950c2a44b66a228e0e8757a3828 P a255e41a83d6499ea839047c801780cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:31:08.996555 9713 ts_tablet_manager.cc:1918] T 947c3950c2a44b66a228e0e8757a3828 P 40a5a24cba3b489885e1df5b3febda4d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:31:08.996604 9713 log.cc:1196] T 947c3950c2a44b66a228e0e8757a3828 P 40a5a24cba3b489885e1df5b3febda4d: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/ts-1/wal/wals/947c3950c2a44b66a228e0e8757a3828
I20260806 06:31:08.996827 9713 ts_tablet_manager.cc:1939] T 947c3950c2a44b66a228e0e8757a3828 P 40a5a24cba3b489885e1df5b3febda4d: Deleting consensus metadata
I20260806 06:31:08.997139 8877 catalog_manager.cc:4841] TS 40a5a24cba3b489885e1df5b3febda4d (127.7.43.66:37701): tablet 947c3950c2a44b66a228e0e8757a3828 (table test-table [id=48bea342222e4334a93e40f61161816f]) successfully deleted
I20260806 06:31:08.998636 9712 ts_tablet_manager.cc:1918] T 947c3950c2a44b66a228e0e8757a3828 P a255e41a83d6499ea839047c801780cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:31:08.998756 9712 log.cc:1196] T 947c3950c2a44b66a228e0e8757a3828 P a255e41a83d6499ea839047c801780cb: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785997861828271-7341-0/minicluster-data/ts-0/wal/wals/947c3950c2a44b66a228e0e8757a3828
I20260806 06:31:08.999008 9712 ts_tablet_manager.cc:1939] T 947c3950c2a44b66a228e0e8757a3828 P a255e41a83d6499ea839047c801780cb: Deleting consensus metadata
I20260806 06:31:08.999320 8876 catalog_manager.cc:4841] TS a255e41a83d6499ea839047c801780cb (127.7.43.65:44971): tablet 947c3950c2a44b66a228e0e8757a3828 (table test-table [id=48bea342222e4334a93e40f61161816f]) successfully deleted
Aug 06 06:31:08 dist-test-slave-6zbq krb5kdc[8837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997868, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.43.65@KRBTEST.COM
I20260806 06:31:09.009234 9609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:08.993017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39242 (local address 127.7.43.65:44971)
0806 06:31:08.993065 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:08.993067 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:08.999558 (+ 6491us) server_negotiation.cc:407] Connection header received
0806 06:31:08.999652 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:08.999653 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:08.999664 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:08.999691 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:09.000445 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:09.001180 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:09.002264 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:09.002581 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:09.002618 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:09.006054 (+ 3436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:09.006062 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:09.006062 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:09.006070 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:09.007986 (+ 1916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:09.008400 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:09.008403 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:09.008403 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:09.008470 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:09.008675 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:09.008675 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:09.008676 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:09.008835 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:09.008974 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:09.009053 (+ 79us) server_negotiation.cc:299] Negotiation successful
0806 06:31:09.009132 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
W20260806 06:31:09.009492 9150 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39242
I20260806 06:31:09.021250 8968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:09.010646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47516 (local address 127.7.43.126:36741)
0806 06:31:09.010701 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:09.010703 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:09.014534 (+ 3831us) server_negotiation.cc:407] Connection header received
0806 06:31:09.014590 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:09.014592 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:09.014604 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:09.014665 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:09.015236 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:09.015989 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:09.016890 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:09.017193 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:09.017259 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:09.018604 (+ 1345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:09.018606 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:09.018606 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:09.018614 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:09.020157 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:09.020530 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:09.020531 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:09.020531 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:09.020586 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:09.020761 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:09.020762 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:09.020762 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:09.020883 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:09.020972 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:09.021075 (+ 103us) server_negotiation.cc:299] Negotiation successful
0806 06:31:09.021163 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
W20260806 06:31:09.021742 8895 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47516
I20260806 06:31:09.022629 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 8998
I20260806 06:31:09.038558 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 9198
I20260806 06:31:09.046267 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 9396
I20260806 06:31:09.055908 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 8854
2026-08-06T06:31:09Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2591 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:31:09 dist-test-slave-6zbq krb5kdc[9725](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:31:09 dist-test-slave-6zbq krb5kdc[9725](info): set up 2 sockets
Aug 06 06:31:09 dist-test-slave-6zbq krb5kdc[9725](info): commencing operation
krb5kdc: starting...
W20260806 06:31:10.166723 7341 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Aug 06 06:31:10 dist-test-slave-6zbq krb5kdc[9725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997870, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:31:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:31:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.43.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
I20260806 06:31:10.272676 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:39573
--webserver_interface=127.7.43.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:39895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.43.126:39573
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:31:10.351785 9743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:10.351962 9743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:10.351984 9743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:10.353070 9743 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:31:10.353120 9743 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 06:31:10.353132 9743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:10.353156 9743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:10.353180 9743 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:31:10.353201 9743 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:31:10.354633 9743 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:39895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.43.126:39573
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:39573
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.43.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9743
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:10.354844 9743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:10.355037 9743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:10.359412 9752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:10.359421 9749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:10.359500 9743 server_base.cc:1034] running on GCE node
W20260806 06:31:10.359716 9748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:10.359936 9743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:10.360183 9743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:10.361315 9743 hybrid_clock.cc:648] HybridClock initialized: now 1785997870361303 us; error 28 us; skew 500 ppm
Aug 06 06:31:10 dist-test-slave-6zbq krb5kdc[9725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997870, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:10.363511 9743 init.cc:377] Logged in from keytab as kudu/127.7.43.126@KRBTEST.COM (short username kudu)
I20260806 06:31:10.363876 9743 webserver.cc:460] Webserver started at http://127.7.43.126:45297/ using document root <none> and password file <none>
I20260806 06:31:10.364053 9743 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:10.364096 9743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:10.364210 9743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:10.365129 9743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/master-0/data/instance:
uuid: "50cb76ac472c4be8a76c82be91a440b9"
format_stamp: "Formatted at 2026-08-06 06:31:10 on dist-test-slave-6zbq"
server_key: "6d2ceb0ebd3befbec9286befc904a9c5"
server_key_iv: "5da6287ad8afde07de45521e423e20a9"
server_key_version: "encryptionkey@0"
I20260806 06:31:10.365446 9743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/master-0/wal/instance:
uuid: "50cb76ac472c4be8a76c82be91a440b9"
format_stamp: "Formatted at 2026-08-06 06:31:10 on dist-test-slave-6zbq"
server_key: "6d2ceb0ebd3befbec9286befc904a9c5"
server_key_iv: "5da6287ad8afde07de45521e423e20a9"
server_key_version: "encryptionkey@0"
I20260806 06:31:10.366674 9743 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:31:10.367293 9761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:10.367446 9743 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:31:10.367527 9743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "50cb76ac472c4be8a76c82be91a440b9"
format_stamp: "Formatted at 2026-08-06 06:31:10 on dist-test-slave-6zbq"
server_key: "6d2ceb0ebd3befbec9286befc904a9c5"
server_key_iv: "5da6287ad8afde07de45521e423e20a9"
server_key_version: "encryptionkey@0"
I20260806 06:31:10.367583 9743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:10.396441 9743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:10.397997 9743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:10.398094 9743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:10.402283 9743 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.126:39573
I20260806 06:31:10.402390 9854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.126:39573 every 8 connection(s)
I20260806 06:31:10.402637 9743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:31:10.403215 9855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:10.405274 9855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50cb76ac472c4be8a76c82be91a440b9: Bootstrap starting.
I20260806 06:31:10.405810 9855 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 50cb76ac472c4be8a76c82be91a440b9: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:10.406068 9855 log.cc:824] T 00000000000000000000000000000000 P 50cb76ac472c4be8a76c82be91a440b9: Log is configured to *not* fsync() on all Append() calls
I20260806 06:31:10.406626 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 9743
I20260806 06:31:10.406708 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/master-0/wal/instance
I20260806 06:31:10.406729 9855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50cb76ac472c4be8a76c82be91a440b9: No bootstrap required, opened a new log
I20260806 06:31:10.406868 7341 external_mini_cluster.cc:1428] Setting key 4706c1249711c594e30241c5e32e83ef
I20260806 06:31:10.409457 9855 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50cb76ac472c4be8a76c82be91a440b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50cb76ac472c4be8a76c82be91a440b9" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 39573 } }
I20260806 06:31:10.409583 9855 raft_consensus.cc:374] T 00000000000000000000000000000000 P 50cb76ac472c4be8a76c82be91a440b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:10.409606 9855 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50cb76ac472c4be8a76c82be91a440b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50cb76ac472c4be8a76c82be91a440b9, State: Initialized, Role: FOLLOWER
I20260806 06:31:10.409710 9855 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50cb76ac472c4be8a76c82be91a440b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50cb76ac472c4be8a76c82be91a440b9" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 39573 } }
I20260806 06:31:10.409780 9855 raft_consensus.cc:388] T 00000000000000000000000000000000 P 50cb76ac472c4be8a76c82be91a440b9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:31:10.409813 9855 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50cb76ac472c4be8a76c82be91a440b9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:31:10.409859 9855 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50cb76ac472c4be8a76c82be91a440b9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:10.410414 9855 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50cb76ac472c4be8a76c82be91a440b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50cb76ac472c4be8a76c82be91a440b9" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 39573 } }
I20260806 06:31:10.410508 9855 leader_election.cc:302] T 00000000000000000000000000000000 P 50cb76ac472c4be8a76c82be91a440b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 50cb76ac472c4be8a76c82be91a440b9; no voters:
I20260806 06:31:10.410655 9855 leader_election.cc:288] T 00000000000000000000000000000000 P 50cb76ac472c4be8a76c82be91a440b9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:31:10.410682 9861 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 50cb76ac472c4be8a76c82be91a440b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:31:10.410799 9861 raft_consensus.cc:686] T 00000000000000000000000000000000 P 50cb76ac472c4be8a76c82be91a440b9 [term 1 LEADER]: Becoming Leader. State: Replica: 50cb76ac472c4be8a76c82be91a440b9, State: Running, Role: LEADER
I20260806 06:31:10.410877 9861 consensus_queue.cc:237] T 00000000000000000000000000000000 P 50cb76ac472c4be8a76c82be91a440b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50cb76ac472c4be8a76c82be91a440b9" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 39573 } }
I20260806 06:31:10.410898 9855 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50cb76ac472c4be8a76c82be91a440b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:31:10.411186 9861 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50cb76ac472c4be8a76c82be91a440b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "50cb76ac472c4be8a76c82be91a440b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50cb76ac472c4be8a76c82be91a440b9" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 39573 } } }
I20260806 06:31:10.411257 9861 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50cb76ac472c4be8a76c82be91a440b9 [sys.catalog]: This master's current role is: LEADER
I20260806 06:31:10.411377 9861 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50cb76ac472c4be8a76c82be91a440b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50cb76ac472c4be8a76c82be91a440b9. Latest consensus state: current_term: 1 leader_uuid: "50cb76ac472c4be8a76c82be91a440b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50cb76ac472c4be8a76c82be91a440b9" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 39573 } } }
I20260806 06:31:10.411504 9869 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:31:10.411732 9861 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50cb76ac472c4be8a76c82be91a440b9 [sys.catalog]: This master's current role is: LEADER
I20260806 06:31:10.412258 9869 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 06:31:10 dist-test-slave-6zbq krb5kdc[9725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997870, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:31:10.417132 9860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:10.407308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41096 (local address 127.7.43.126:39573)
0806 06:31:10.407465 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:10.407474 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:10.407497 (+ 23us) server_negotiation.cc:407] Connection header received
0806 06:31:10.407915 (+ 418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:10.407921 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:10.407963 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:10.408064 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:10.408818 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:10.409678 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:10.410476 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:10.411058 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:10.411158 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:10.413947 (+ 2789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:10.413951 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:10.413951 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:10.413962 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:10.415710 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:10.416013 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:10.416016 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:10.416016 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:10.416122 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:10.416291 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:10.416292 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:10.416292 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:10.416579 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:10.416736 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:10.416804 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 06:31:10.416932 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":66,"threads_started":1}
I20260806 06:31:10.417251 9869 catalog_manager.cc:1347] Generated new cluster ID: e41c8f8464ca45209b62dba976430278
I20260806 06:31:10.417287 9869 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:31:10.427304 9869 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:31:10.427738 9869 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:31:10.435602 9869 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 50cb76ac472c4be8a76c82be91a440b9: Generated new TSK 0
I20260806 06:31:10.435745 9869 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.43.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab.
Entry for principal kudu/127.7.43.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab.
I20260806 06:31:10.461198 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.65:0
--local_ip_for_outbound_sockets=127.7.43.65
--webserver_interface=127.7.43.65
--webserver_port=0
--tserver_master_addrs=127.7.43.126:39573
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab
--principal=kudu/127.7.43.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:39895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:31:10.541798 9892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:10.541982 9892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:10.542006 9892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:10.543138 9892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:10.543176 9892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:10.543195 9892 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 06:31:10.543206 9892 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 06:31:10.543234 9892 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.65
I20260806 06:31:10.544654 9892 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:39895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab
--principal=kudu/127.7.43.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.43.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.43.126:39573
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9892
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.65
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:10.544879 9892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:10.545069 9892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:10.549589 9899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:10.549594 9902 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:10.549759 9892 server_base.cc:1034] running on GCE node
W20260806 06:31:10.549862 9898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:10.550067 9892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:10.550261 9892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:10.551393 9892 hybrid_clock.cc:648] HybridClock initialized: now 1785997870551381 us; error 36 us; skew 500 ppm
Aug 06 06:31:10 dist-test-slave-6zbq krb5kdc[9725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997870, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:10.553624 9892 init.cc:377] Logged in from keytab as kudu/127.7.43.65@KRBTEST.COM (short username kudu)
I20260806 06:31:10.554028 9892 webserver.cc:460] Webserver started at http://127.7.43.65:39443/ using document root <none> and password file <none>
I20260806 06:31:10.554231 9892 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:10.554277 9892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:10.554421 9892 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:10.555362 9892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-0/data/instance:
uuid: "618e706add47425f93508b6d2ae311cd"
format_stamp: "Formatted at 2026-08-06 06:31:10 on dist-test-slave-6zbq"
server_key: "be659ff4b1fc38c501e6100ee8e44d86"
server_key_iv: "aa770bce31fbb17de65c9f0f3baf9a2c"
server_key_version: "encryptionkey@0"
I20260806 06:31:10.555681 9892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-0/wal/instance:
uuid: "618e706add47425f93508b6d2ae311cd"
format_stamp: "Formatted at 2026-08-06 06:31:10 on dist-test-slave-6zbq"
server_key: "be659ff4b1fc38c501e6100ee8e44d86"
server_key_iv: "aa770bce31fbb17de65c9f0f3baf9a2c"
server_key_version: "encryptionkey@0"
I20260806 06:31:10.556937 9892 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:31:10.557540 9912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:10.557677 9892 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:31:10.557740 9892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-0/wal
uuid: "618e706add47425f93508b6d2ae311cd"
format_stamp: "Formatted at 2026-08-06 06:31:10 on dist-test-slave-6zbq"
server_key: "be659ff4b1fc38c501e6100ee8e44d86"
server_key_iv: "aa770bce31fbb17de65c9f0f3baf9a2c"
server_key_version: "encryptionkey@0"
I20260806 06:31:10.557811 9892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:10.576849 9892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:10.578498 9892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:10.578619 9892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:10.579101 9892 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:31:10.579140 9892 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:10.579161 9892 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:31:10.579173 9892 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 06:31:10 dist-test-slave-6zbq krb5kdc[9725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997870, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.65@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:31:10.585840 9892 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.65:43601
I20260806 06:31:10.585934 10068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.65:43601 every 8 connection(s)
I20260806 06:31:10.586185 9892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
I20260806 06:31:10.588630 9860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:10.579517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:57939 (local address 127.7.43.126:39573)
0806 06:31:10.579581 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:10.579584 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:10.579751 (+ 167us) server_negotiation.cc:407] Connection header received
0806 06:31:10.580245 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:10.580246 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:10.580258 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:10.580453 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:10.581365 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:10.582133 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:10.583605 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:10.583829 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:10.583894 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:10.586251 (+ 2357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:10.586253 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:10.586253 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:10.586262 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:10.587638 (+ 1376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:10.587967 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:10.587968 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:10.587968 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:10.588011 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:10.588182 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:10.588182 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:10.588183 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:10.588277 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:10.588321 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:10.588507 (+ 186us) server_negotiation.cc:299] Negotiation successful
0806 06:31:10.588567 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 06:31:10.588794 9922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:10.579474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:39573 (local address 127.7.43.65:57939)
0806 06:31:10.579627 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:10.579665 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:10.580150 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:10.580591 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:10.580596 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:10.580941 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:10.581251 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:10.581257 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:10.582238 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:10.582239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:10.583504 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:10.583508 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:10.583900 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:10.583901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:10.584045 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:10.584484 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:10.584496 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:10.586160 (+ 1664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:10.587703 (+ 1543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:10.587707 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:10.587708 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:10.587911 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:10.588053 (+ 142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:10.588053 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:10.588054 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:10.588138 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:10.588318 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:10.588321 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:10.588423 (+ 102us) client_negotiation.cc:769] Sending connection context
0806 06:31:10.588495 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 06:31:10.588606 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":85,"threads_started":1}
I20260806 06:31:10.589278 10069 heartbeater.cc:347] Connected to a master server at 127.7.43.126:39573
I20260806 06:31:10.589359 10069 heartbeater.cc:464] Registering TS with master...
I20260806 06:31:10.589537 10069 heartbeater.cc:511] Master 127.7.43.126:39573 requested a full tablet report, sending...
I20260806 06:31:10.590045 9790 ts_manager.cc:194] Registered new tserver with Master: 618e706add47425f93508b6d2ae311cd (127.7.43.65:43601)
I20260806 06:31:10.590833 9790 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.65@KRBTEST.COM'} at 127.7.43.65:57939
I20260806 06:31:10.594782 9860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:10.590237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:59591 (local address 127.7.43.126:39573)
0806 06:31:10.590274 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:10.590276 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:10.590401 (+ 125us) server_negotiation.cc:407] Connection header received
0806 06:31:10.590466 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:10.590467 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:10.590475 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:10.590525 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:10.590769 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:10.591357 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:10.592032 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:10.592220 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:10.592264 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:10.592925 (+ 661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:10.592926 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:10.592926 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:10.592931 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:10.594052 (+ 1121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:10.594281 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:10.594281 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:10.594281 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:10.594321 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:10.594463 (+ 142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:10.594463 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:10.594463 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:10.594547 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:10.594589 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:10.594681 (+ 92us) server_negotiation.cc:299] Negotiation successful
0806 06:31:10.594731 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:31:10.594790 10078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:10.590186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:39573 (local address 127.7.43.65:59591)
0806 06:31:10.590322 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:10.590332 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:10.590413 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:10.590529 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:10.590529 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:10.590667 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:10.590718 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:10.590719 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:10.591448 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:10.591448 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:10.591966 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:10.591968 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:10.592269 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:10.592270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:10.592319 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:10.592615 (+ 296us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:10.592618 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:10.592864 (+ 246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:10.594108 (+ 1244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:10.594108 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:10.594108 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:10.594233 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:10.594380 (+ 147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:10.594380 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:10.594380 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:10.594425 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:10.594586 (+ 161us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:10.594586 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:10.594618 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 06:31:10.594659 (+ 41us) client_negotiation.cc:240] Negotiation successful
0806 06:31:10.594726 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":88,"threads_started":1}
I20260806 06:31:10.595638 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 9892
I20260806 06:31:10.595738 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-0/wal/instance
I20260806 06:31:10.595914 7341 external_mini_cluster.cc:1428] Setting key 944fb5de9bd612ef2bcc3a24c2ce67ac
WARNING: no policy specified for kudu/127.7.43.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab.
Entry for principal kudu/127.7.43.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab.
I20260806 06:31:10.619115 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.66:0
--local_ip_for_outbound_sockets=127.7.43.66
--webserver_interface=127.7.43.66
--webserver_port=0
--tserver_master_addrs=127.7.43.126:39573
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab
--principal=kudu/127.7.43.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:39895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:31:10.697638 10082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:10.697813 10082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:10.697835 10082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:10.698951 10082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:10.698988 10082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:10.699007 10082 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 06:31:10.699029 10082 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 06:31:10.699059 10082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.66
I20260806 06:31:10.700433 10082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:39895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab
--principal=kudu/127.7.43.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.43.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.43.126:39573
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10082
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.66
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:10.700637 10082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:10.700825 10082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:31:10.705382 10082 server_base.cc:1034] running on GCE node
W20260806 06:31:10.705341 10089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:10.705355 10093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:10.705353 10088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:10.705736 10082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:10.705943 10082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:10.707067 10082 hybrid_clock.cc:648] HybridClock initialized: now 1785997870707057 us; error 27 us; skew 500 ppm
Aug 06 06:31:10 dist-test-slave-6zbq krb5kdc[9725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997870, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:10.709239 10082 init.cc:377] Logged in from keytab as kudu/127.7.43.66@KRBTEST.COM (short username kudu)
I20260806 06:31:10.709663 10082 webserver.cc:460] Webserver started at http://127.7.43.66:34687/ using document root <none> and password file <none>
I20260806 06:31:10.709854 10082 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:10.709899 10082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:10.710021 10082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:10.711205 10082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-1/data/instance:
uuid: "a73ce5908de74a62bb9267d0b1651c72"
format_stamp: "Formatted at 2026-08-06 06:31:10 on dist-test-slave-6zbq"
server_key: "644c3ce85f8ad83001083d9f14f93aa2"
server_key_iv: "443732ae4a82c67cccc6e85097300913"
server_key_version: "encryptionkey@0"
I20260806 06:31:10.711522 10082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-1/wal/instance:
uuid: "a73ce5908de74a62bb9267d0b1651c72"
format_stamp: "Formatted at 2026-08-06 06:31:10 on dist-test-slave-6zbq"
server_key: "644c3ce85f8ad83001083d9f14f93aa2"
server_key_iv: "443732ae4a82c67cccc6e85097300913"
server_key_version: "encryptionkey@0"
I20260806 06:31:10.712798 10082 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:31:10.713409 10104 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:10.713557 10082 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:31:10.713639 10082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-1/wal
uuid: "a73ce5908de74a62bb9267d0b1651c72"
format_stamp: "Formatted at 2026-08-06 06:31:10 on dist-test-slave-6zbq"
server_key: "644c3ce85f8ad83001083d9f14f93aa2"
server_key_iv: "443732ae4a82c67cccc6e85097300913"
server_key_version: "encryptionkey@0"
I20260806 06:31:10.713697 10082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:10.747253 10082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:10.748914 10082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:10.749033 10082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:10.749490 10082 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:31:10.749531 10082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:10.749557 10082 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:31:10.749574 10082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 06:31:10 dist-test-slave-6zbq krb5kdc[9725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997870, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.66@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:31:10.756325 10082 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.66:34453
I20260806 06:31:10.756500 10259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.66:34453 every 8 connection(s)
I20260806 06:31:10.756691 10082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
I20260806 06:31:10.758654 9860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:10.749856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:39929 (local address 127.7.43.126:39573)
0806 06:31:10.749916 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:10.749919 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:10.750091 (+ 172us) server_negotiation.cc:407] Connection header received
0806 06:31:10.750595 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:10.750596 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:10.750611 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:10.750685 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:10.751250 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:10.752012 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:10.752968 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:10.753226 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:10.753283 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:10.756203 (+ 2920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:10.756205 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:10.756206 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:10.756214 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:10.757640 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:10.757949 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:10.757949 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:10.757950 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:10.757996 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:10.758154 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:10.758154 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:10.758154 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:10.758243 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:10.758286 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:10.758532 (+ 246us) server_negotiation.cc:299] Negotiation successful
0806 06:31:10.758592 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 06:31:10.758791 10114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:10.749822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:39573 (local address 127.7.43.66:39929)
0806 06:31:10.749974 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:10.750011 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:10.750502 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:10.750739 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:10.750743 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:10.750956 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:10.751166 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:10.751171 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:10.752098 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:10.752099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:10.752883 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:10.752885 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:10.753299 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:10.753300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:10.753406 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:10.753825 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:10.753845 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:10.756102 (+ 2257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:10.757700 (+ 1598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:10.757703 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:10.757704 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:10.757892 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:10.758036 (+ 144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:10.758037 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:10.758037 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:10.758118 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:10.758281 (+ 163us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:10.758283 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:10.758423 (+ 140us) client_negotiation.cc:769] Sending connection context
0806 06:31:10.758508 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 06:31:10.758617 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":84,"threads_started":1}
I20260806 06:31:10.759231 10261 heartbeater.cc:347] Connected to a master server at 127.7.43.126:39573
I20260806 06:31:10.759311 10261 heartbeater.cc:464] Registering TS with master...
I20260806 06:31:10.759470 10261 heartbeater.cc:511] Master 127.7.43.126:39573 requested a full tablet report, sending...
I20260806 06:31:10.759850 9788 ts_manager.cc:194] Registered new tserver with Master: a73ce5908de74a62bb9267d0b1651c72 (127.7.43.66:34453)
I20260806 06:31:10.760334 9788 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.66@KRBTEST.COM'} at 127.7.43.66:39929
I20260806 06:31:10.763255 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 10082
I20260806 06:31:10.763330 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-1/wal/instance
I20260806 06:31:10.763482 7341 external_mini_cluster.cc:1428] Setting key 4e6616c275a0f21a2b2217b53ed31088
I20260806 06:31:10.765301 9860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:10.760168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:36183 (local address 127.7.43.126:39573)
0806 06:31:10.760455 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:10.760456 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:10.760476 (+ 20us) server_negotiation.cc:407] Connection header received
0806 06:31:10.760493 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:10.760494 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:10.760502 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:10.760553 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:10.760873 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:10.761314 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:10.761929 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:10.762092 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:10.762133 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:10.762837 (+ 704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:10.762838 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:10.762838 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:10.762844 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:10.764255 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:10.764552 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:10.764553 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:10.764553 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:10.764604 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:10.764764 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:10.764764 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:10.764765 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:10.764944 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:10.765003 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:10.765145 (+ 142us) server_negotiation.cc:299] Negotiation successful
0806 06:31:10.765210 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272}
I20260806 06:31:10.765308 10273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:10.760116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:39573 (local address 127.7.43.66:36183)
0806 06:31:10.760250 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:10.760260 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:10.760311 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:10.760571 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:10.760572 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:10.760749 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:10.760801 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:10.760803 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:10.761400 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:10.761400 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:10.761874 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:10.761876 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:10.762151 (+ 275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:10.762151 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:10.762201 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:10.762537 (+ 336us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:10.762540 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:10.762774 (+ 234us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:10.764326 (+ 1552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:10.764326 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:10.764327 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:10.764492 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:10.764655 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:10.764656 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:10.764656 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:10.764712 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:10.765009 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:10.765010 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:10.765054 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 06:31:10.765116 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 06:31:10.765212 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":67,"threads_started":1}
WARNING: no policy specified for kudu/127.7.43.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab.
Entry for principal kudu/127.7.43.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab.
I20260806 06:31:10.786094 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.67:0
--local_ip_for_outbound_sockets=127.7.43.67
--webserver_interface=127.7.43.67
--webserver_port=0
--tserver_master_addrs=127.7.43.126:39573
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab
--principal=kudu/127.7.43.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:39895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:31:10.863507 10279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:10.863680 10279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:10.863703 10279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:10.864748 10279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:10.864781 10279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:10.864799 10279 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 06:31:10.864815 10279 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 06:31:10.864830 10279 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.67
I20260806 06:31:10.866160 10279 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:39895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab
--principal=kudu/127.7.43.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.43.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.43.126:39573
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10279
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.67
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:10.866319 10279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:10.866505 10279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:10.870927 10288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:10.870991 10287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:10.870952 10291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:10.871083 10279 server_base.cc:1034] running on GCE node
I20260806 06:31:10.871287 10279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:10.871483 10279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:10.872622 10279 hybrid_clock.cc:648] HybridClock initialized: now 1785997870872605 us; error 27 us; skew 500 ppm
Aug 06 06:31:10 dist-test-slave-6zbq krb5kdc[9725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997870, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:10.874810 10279 init.cc:377] Logged in from keytab as kudu/127.7.43.67@KRBTEST.COM (short username kudu)
I20260806 06:31:10.875200 10279 webserver.cc:460] Webserver started at http://127.7.43.67:33287/ using document root <none> and password file <none>
I20260806 06:31:10.875387 10279 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:10.875432 10279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:10.875551 10279 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:10.876470 10279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-2/data/instance:
uuid: "5648ea51493544a989bf87593657c2dd"
format_stamp: "Formatted at 2026-08-06 06:31:10 on dist-test-slave-6zbq"
server_key: "46b4a4ae9e00d8521c046e6734728e0a"
server_key_iv: "ee9bd1d35fcb4a93381d87b9ebb929b7"
server_key_version: "encryptionkey@0"
I20260806 06:31:10.876780 10279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-2/wal/instance:
uuid: "5648ea51493544a989bf87593657c2dd"
format_stamp: "Formatted at 2026-08-06 06:31:10 on dist-test-slave-6zbq"
server_key: "46b4a4ae9e00d8521c046e6734728e0a"
server_key_iv: "ee9bd1d35fcb4a93381d87b9ebb929b7"
server_key_version: "encryptionkey@0"
I20260806 06:31:10.877915 10279 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:31:10.878592 10303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:10.878746 10279 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:31:10.878826 10279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-2/wal
uuid: "5648ea51493544a989bf87593657c2dd"
format_stamp: "Formatted at 2026-08-06 06:31:10 on dist-test-slave-6zbq"
server_key: "46b4a4ae9e00d8521c046e6734728e0a"
server_key_iv: "ee9bd1d35fcb4a93381d87b9ebb929b7"
server_key_version: "encryptionkey@0"
I20260806 06:31:10.878885 10279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:10.890055 10279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:10.892052 10279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:10.892184 10279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:10.892632 10279 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:31:10.892671 10279 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:10.892696 10279 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:31:10.892726 10279 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 06:31:10 dist-test-slave-6zbq krb5kdc[9725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997870, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.67@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:31:10.899428 10279 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.67:33315
I20260806 06:31:10.899498 10465 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.67:33315 every 8 connection(s)
I20260806 06:31:10.899747 10279 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
I20260806 06:31:10.899966 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 10279
I20260806 06:31:10.900063 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-2/wal/instance
I20260806 06:31:10.900257 7341 external_mini_cluster.cc:1428] Setting key 6c9e8e84b42af278362e444d1e58a420
I20260806 06:31:10.901295 9860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:10.892968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:48501 (local address 127.7.43.126:39573)
0806 06:31:10.893024 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:10.893027 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:10.893217 (+ 190us) server_negotiation.cc:407] Connection header received
0806 06:31:10.893636 (+ 419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:10.893637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:10.893650 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:10.893725 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:10.894322 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:10.894884 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:10.895803 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:10.896002 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:10.896057 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:10.898383 (+ 2326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:10.898385 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:10.898385 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:10.898392 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:10.899920 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:10.900351 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:10.900352 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:10.900353 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:10.900409 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:10.900693 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:10.900694 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:10.900695 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:10.900815 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:10.900881 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:10.901153 (+ 272us) server_negotiation.cc:299] Negotiation successful
0806 06:31:10.901234 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 06:31:10.901386 10318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:10.892958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:39573 (local address 127.7.43.67:48501)
0806 06:31:10.893107 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:10.893143 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:10.893545 (+ 402us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:10.893771 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:10.893773 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:10.894024 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:10.894229 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:10.894234 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:10.894967 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:10.894968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:10.895719 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:10.895722 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:10.896070 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:10.896071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:10.896195 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:10.896639 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:10.896648 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:10.898276 (+ 1628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:10.900010 (+ 1734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:10.900018 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:10.900020 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:10.900287 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:10.900493 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:10.900494 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:10.900495 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:10.900633 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:10.900882 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:10.900887 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:10.901005 (+ 118us) client_negotiation.cc:769] Sending connection context
0806 06:31:10.901115 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 06:31:10.901226 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":76,"threads_started":1}
I20260806 06:31:10.901793 10467 heartbeater.cc:347] Connected to a master server at 127.7.43.126:39573
I20260806 06:31:10.901873 10467 heartbeater.cc:464] Registering TS with master...
I20260806 06:31:10.902056 10467 heartbeater.cc:511] Master 127.7.43.126:39573 requested a full tablet report, sending...
I20260806 06:31:10.902407 9790 ts_manager.cc:194] Registered new tserver with Master: 5648ea51493544a989bf87593657c2dd (127.7.43.67:33315)
I20260806 06:31:10.902808 9790 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.67@KRBTEST.COM'} at 127.7.43.67:48501
I20260806 06:31:10.907217 9860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:10.902792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:51587 (local address 127.7.43.126:39573)
0806 06:31:10.902827 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:10.902829 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:10.902916 (+ 87us) server_negotiation.cc:407] Connection header received
0806 06:31:10.903040 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:10.903041 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:10.903048 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:10.903095 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:10.903375 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:10.903863 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:10.904479 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:10.904638 (+ 159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:10.904677 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:10.905323 (+ 646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:10.905323 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:10.905324 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:10.905329 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:10.906482 (+ 1153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:10.906755 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:10.906755 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:10.906756 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:10.906809 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:10.906921 (+ 112us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:10.906922 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:10.906922 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:10.906999 (+ 77us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:10.907037 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:10.907117 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 06:31:10.907166 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:31:10.907235 10477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:10.902707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:39573 (local address 127.7.43.67:51587)
0806 06:31:10.902827 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:10.902838 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:10.902975 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:10.903100 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:10.903101 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:10.903260 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:10.903313 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:10.903314 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:10.903931 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:10.903931 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:10.904428 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:10.904429 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:10.904683 (+ 254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:10.904683 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:10.904735 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:10.905018 (+ 283us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:10.905022 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:10.905262 (+ 240us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:10.906545 (+ 1283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:10.906546 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:10.906547 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:10.906698 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:10.906850 (+ 152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:10.906850 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:10.906850 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:10.906890 (+ 40us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:10.907029 (+ 139us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:10.907030 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:10.907061 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 06:31:10.907102 (+ 41us) client_negotiation.cc:240] Negotiation successful
0806 06:31:10.907171 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":65,"threads_started":1}
I20260806 06:31:10.911810 7341 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 06:31:10 dist-test-slave-6zbq krb5kdc[9725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997870, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 06:31:10 dist-test-slave-6zbq krb5kdc[9725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997870, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:31:10.932803 9860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:10.927254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41102 (local address 127.7.43.126:39573)
0806 06:31:10.927300 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:10.927302 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:10.927309 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:31:10.927357 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:10.927358 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:10.927366 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:10.927408 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:10.927868 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:10.928302 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:10.928958 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:10.929125 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:10.929176 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:10.931065 (+ 1889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:10.931066 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:10.931066 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:10.931072 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:10.932060 (+ 988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:10.932319 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:10.932319 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:10.932320 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:10.932358 (+ 38us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:10.932486 (+ 128us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:10.932486 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:10.932486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:10.932563 (+ 77us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:10.932626 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:10.932691 (+ 65us) server_negotiation.cc:299] Negotiation successful
0806 06:31:10.932749 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 06:31:10.934360 9790 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41102:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:31:10.936395 9790 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:31:10.942310 10497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:10.938757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40366 (local address 127.7.43.65:43601)
0806 06:31:10.938894 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:10.938899 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:10.938909 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:31:10.938946 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:10.938948 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:10.938963 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:10.939187 (+ 224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:31:10.939325 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:10.940013 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:10.941334 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:10.941924 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:10.941983 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:10.942039 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:10.942171 (+ 132us) server_negotiation.cc:299] Negotiation successful
0806 06:31:10.942214 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":74,"threads_started":1}
I20260806 06:31:10.942297 10490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:10.938440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.66:34453 (local address 127.0.0.1:53890)
0806 06:31:10.938525 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:10.938562 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:10.938640 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:10.938836 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:10.938838 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:10.938840 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:31:10.938897 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:10.938898 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:10.940383 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:10.940384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:10.941235 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:10.941238 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:10.942028 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:10.942029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:10.942105 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:10.942110 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:31:10.942139 (+ 29us) client_negotiation.cc:240] Negotiation successful
0806 06:31:10.942212 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62,"thread_start_us":42,"threads_started":1}
I20260806 06:31:10.942304 10496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:10.938441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:43601 (local address 127.0.0.1:40366)
0806 06:31:10.938713 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:10.938731 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:10.938781 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:10.939062 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:10.939063 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:10.939065 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:31:10.939114 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:10.939115 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:10.940092 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:10.940093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:10.941240 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:10.941243 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:10.941997 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:10.941998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:10.942076 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:10.942088 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 06:31:10.942149 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 06:31:10.942216 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"mutex_wait_us":30,"thread_start_us":74,"threads_started":1}
I20260806 06:31:10.942744 10494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:10.938618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.67:33315 (local address 127.0.0.1:60292)
0806 06:31:10.938982 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:10.938992 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:10.939030 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:10.939310 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:10.939311 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:10.939313 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:31:10.939349 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:10.939350 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:10.940488 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:10.940489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:10.941535 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:10.941538 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:10.942511 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:10.942512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:10.942580 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:10.942584 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:31:10.942642 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 06:31:10.942679 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":59,"threads_started":1}
I20260806 06:31:10.942961 10498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:10.938971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60292 (local address 127.7.43.67:33315)
0806 06:31:10.939124 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:10.939128 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:10.939136 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:31:10.939166 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:10.939168 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:10.939181 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:10.939255 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:31:10.939416 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:10.940412 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:10.941650 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:10.942320 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:10.942416 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:10.942478 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:10.942845 (+ 367us) server_negotiation.cc:299] Negotiation successful
0806 06:31:10.942881 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":82,"threads_started":1}
I20260806 06:31:10.943255 9975 tablet_service.cc:1467] Processing CreateTablet for tablet 0536db0542ac4ec881d8f81b7db102cf (DEFAULT_TABLE table=test-table [id=854097bba19f4db3b13348624195986f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:31:10.943383 10495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:10.938523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53890 (local address 127.7.43.66:34453)
0806 06:31:10.938695 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:10.938699 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:10.938708 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:31:10.938744 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:10.938746 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:10.938760 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:10.938828 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:31:10.939148 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:10.940284 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:10.941324 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:10.941932 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:10.942927 (+ 995us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:10.942992 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:10.943257 (+ 265us) server_negotiation.cc:299] Negotiation successful
0806 06:31:10.943287 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":73,"threads_started":1}
I20260806 06:31:10.943622 9975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0536db0542ac4ec881d8f81b7db102cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:10.943542 10371 tablet_service.cc:1467] Processing CreateTablet for tablet 0536db0542ac4ec881d8f81b7db102cf (DEFAULT_TABLE table=test-table [id=854097bba19f4db3b13348624195986f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:31:10.943841 10371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0536db0542ac4ec881d8f81b7db102cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:10.946866 10173 tablet_service.cc:1467] Processing CreateTablet for tablet 0536db0542ac4ec881d8f81b7db102cf (DEFAULT_TABLE table=test-table [id=854097bba19f4db3b13348624195986f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:31:10.947187 10173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0536db0542ac4ec881d8f81b7db102cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:10.947300 10501 tablet_bootstrap.cc:492] T 0536db0542ac4ec881d8f81b7db102cf P 618e706add47425f93508b6d2ae311cd: Bootstrap starting.
I20260806 06:31:10.947397 10499 tablet_bootstrap.cc:492] T 0536db0542ac4ec881d8f81b7db102cf P 5648ea51493544a989bf87593657c2dd: Bootstrap starting.
I20260806 06:31:10.947970 10501 tablet_bootstrap.cc:654] T 0536db0542ac4ec881d8f81b7db102cf P 618e706add47425f93508b6d2ae311cd: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:10.948140 10499 tablet_bootstrap.cc:654] T 0536db0542ac4ec881d8f81b7db102cf P 5648ea51493544a989bf87593657c2dd: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:10.948292 10501 log.cc:824] T 0536db0542ac4ec881d8f81b7db102cf P 618e706add47425f93508b6d2ae311cd: Log is configured to *not* fsync() on all Append() calls
I20260806 06:31:10.948462 10499 log.cc:824] T 0536db0542ac4ec881d8f81b7db102cf P 5648ea51493544a989bf87593657c2dd: Log is configured to *not* fsync() on all Append() calls
I20260806 06:31:10.949059 10501 tablet_bootstrap.cc:492] T 0536db0542ac4ec881d8f81b7db102cf P 618e706add47425f93508b6d2ae311cd: No bootstrap required, opened a new log
I20260806 06:31:10.949113 10501 ts_tablet_manager.cc:1397] T 0536db0542ac4ec881d8f81b7db102cf P 618e706add47425f93508b6d2ae311cd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:31:10.949242 10499 tablet_bootstrap.cc:492] T 0536db0542ac4ec881d8f81b7db102cf P 5648ea51493544a989bf87593657c2dd: No bootstrap required, opened a new log
I20260806 06:31:10.949359 10499 ts_tablet_manager.cc:1397] T 0536db0542ac4ec881d8f81b7db102cf P 5648ea51493544a989bf87593657c2dd: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:31:10.950486 10505 tablet_bootstrap.cc:492] T 0536db0542ac4ec881d8f81b7db102cf P a73ce5908de74a62bb9267d0b1651c72: Bootstrap starting.
I20260806 06:31:10.950714 10501 raft_consensus.cc:348] T 0536db0542ac4ec881d8f81b7db102cf P 618e706add47425f93508b6d2ae311cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } }
I20260806 06:31:10.950886 10501 raft_consensus.cc:374] T 0536db0542ac4ec881d8f81b7db102cf P 618e706add47425f93508b6d2ae311cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:10.950925 10501 raft_consensus.cc:729] T 0536db0542ac4ec881d8f81b7db102cf P 618e706add47425f93508b6d2ae311cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 618e706add47425f93508b6d2ae311cd, State: Initialized, Role: FOLLOWER
I20260806 06:31:10.951025 10501 consensus_queue.cc:260] T 0536db0542ac4ec881d8f81b7db102cf P 618e706add47425f93508b6d2ae311cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } }
I20260806 06:31:10.951105 10505 tablet_bootstrap.cc:654] T 0536db0542ac4ec881d8f81b7db102cf P a73ce5908de74a62bb9267d0b1651c72: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:10.951308 10501 ts_tablet_manager.cc:1428] T 0536db0542ac4ec881d8f81b7db102cf P 618e706add47425f93508b6d2ae311cd: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:31:10.951323 10069 heartbeater.cc:503] Master 127.7.43.126:39573 was elected leader, sending a full tablet report...
I20260806 06:31:10.951411 10505 log.cc:824] T 0536db0542ac4ec881d8f81b7db102cf P a73ce5908de74a62bb9267d0b1651c72: Log is configured to *not* fsync() on all Append() calls
I20260806 06:31:10.951339 10499 raft_consensus.cc:348] T 0536db0542ac4ec881d8f81b7db102cf P 5648ea51493544a989bf87593657c2dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } }
I20260806 06:31:10.951473 10499 raft_consensus.cc:374] T 0536db0542ac4ec881d8f81b7db102cf P 5648ea51493544a989bf87593657c2dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:10.951502 10499 raft_consensus.cc:729] T 0536db0542ac4ec881d8f81b7db102cf P 5648ea51493544a989bf87593657c2dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5648ea51493544a989bf87593657c2dd, State: Initialized, Role: FOLLOWER
I20260806 06:31:10.951611 10499 consensus_queue.cc:260] T 0536db0542ac4ec881d8f81b7db102cf P 5648ea51493544a989bf87593657c2dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } }
I20260806 06:31:10.951881 10467 heartbeater.cc:503] Master 127.7.43.126:39573 was elected leader, sending a full tablet report...
I20260806 06:31:10.951864 10499 ts_tablet_manager.cc:1428] T 0536db0542ac4ec881d8f81b7db102cf P 5648ea51493544a989bf87593657c2dd: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:31:10.952109 10505 tablet_bootstrap.cc:492] T 0536db0542ac4ec881d8f81b7db102cf P a73ce5908de74a62bb9267d0b1651c72: No bootstrap required, opened a new log
I20260806 06:31:10.952183 10505 ts_tablet_manager.cc:1397] T 0536db0542ac4ec881d8f81b7db102cf P a73ce5908de74a62bb9267d0b1651c72: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:31:10.953636 10505 raft_consensus.cc:348] T 0536db0542ac4ec881d8f81b7db102cf P a73ce5908de74a62bb9267d0b1651c72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } }
I20260806 06:31:10.953759 10505 raft_consensus.cc:374] T 0536db0542ac4ec881d8f81b7db102cf P a73ce5908de74a62bb9267d0b1651c72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:10.953799 10505 raft_consensus.cc:729] T 0536db0542ac4ec881d8f81b7db102cf P a73ce5908de74a62bb9267d0b1651c72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a73ce5908de74a62bb9267d0b1651c72, State: Initialized, Role: FOLLOWER
I20260806 06:31:10.953904 10505 consensus_queue.cc:260] T 0536db0542ac4ec881d8f81b7db102cf P a73ce5908de74a62bb9267d0b1651c72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } }
I20260806 06:31:10.954139 10505 ts_tablet_manager.cc:1428] T 0536db0542ac4ec881d8f81b7db102cf P a73ce5908de74a62bb9267d0b1651c72: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:31:10.954183 10261 heartbeater.cc:503] Master 127.7.43.126:39573 was elected leader, sending a full tablet report...
W20260806 06:31:11.007207 10263 tablet.cc:2369] T 0536db0542ac4ec881d8f81b7db102cf P a73ce5908de74a62bb9267d0b1651c72: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 06:31:11.086845 10070 tablet.cc:2369] T 0536db0542ac4ec881d8f81b7db102cf P 618e706add47425f93508b6d2ae311cd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:31:11.092456 10510 raft_consensus.cc:482] T 0536db0542ac4ec881d8f81b7db102cf P 5648ea51493544a989bf87593657c2dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:31:11.092564 10510 raft_consensus.cc:504] T 0536db0542ac4ec881d8f81b7db102cf P 5648ea51493544a989bf87593657c2dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } }
I20260806 06:31:11.092885 10510 leader_election.cc:288] T 0536db0542ac4ec881d8f81b7db102cf P 5648ea51493544a989bf87593657c2dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 618e706add47425f93508b6d2ae311cd (127.7.43.65:43601), a73ce5908de74a62bb9267d0b1651c72 (127.7.43.66:34453)
I20260806 06:31:11.095878 10497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:11.093109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:40745 (local address 127.7.43.65:43601)
0806 06:31:11.093155 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:11.093157 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:11.093169 (+ 12us) server_negotiation.cc:407] Connection header received
0806 06:31:11.093253 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:11.093254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:11.093274 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:11.093340 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:31:11.093525 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:11.094028 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:11.094982 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:11.095539 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:11.095612 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:11.095624 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:11.095766 (+ 142us) server_negotiation.cc:299] Negotiation successful
0806 06:31:11.095808 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 06:31:11.095878 10495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:11.093124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:47201 (local address 127.7.43.66:34453)
0806 06:31:11.093163 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:11.093165 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:11.093338 (+ 173us) server_negotiation.cc:407] Connection header received
0806 06:31:11.093412 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:11.093413 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:11.093427 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:11.093484 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:31:11.093633 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:11.094113 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:11.095046 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:11.095565 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:11.095622 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:11.095630 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:11.095768 (+ 138us) server_negotiation.cc:299] Negotiation successful
0806 06:31:11.095807 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:31:11.095890 10318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:11.093062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:43601 (local address 127.7.43.67:40745)
0806 06:31:11.093123 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:11.093134 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:11.093176 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:11.093380 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:11.093381 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:11.093383 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:31:11.093460 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:11.093462 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:11.094100 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:11.094101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:11.094882 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:11.094885 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:11.095626 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:11.095628 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:11.095697 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:11.095702 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:31:11.095753 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 06:31:11.095797 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260806 06:31:11.095897 10514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:11.093062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.66:34453 (local address 127.7.43.67:47201)
0806 06:31:11.093258 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:11.093268 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:11.093334 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:11.093490 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:11.093491 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:11.093492 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:31:11.093572 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:11.093574 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:11.094186 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:11.094187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:11.094962 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:11.094965 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:11.095644 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:11.095645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:11.095707 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:11.095710 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:31:11.095756 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 06:31:11.095791 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":13,"thread_start_us":118,"threads_started":1}
I20260806 06:31:11.096431 10197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0536db0542ac4ec881d8f81b7db102cf" candidate_uuid: "5648ea51493544a989bf87593657c2dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a73ce5908de74a62bb9267d0b1651c72" is_pre_election: true
I20260806 06:31:11.096459 10002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0536db0542ac4ec881d8f81b7db102cf" candidate_uuid: "5648ea51493544a989bf87593657c2dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "618e706add47425f93508b6d2ae311cd" is_pre_election: true
I20260806 06:31:11.096563 10197 raft_consensus.cc:2454] T 0536db0542ac4ec881d8f81b7db102cf P a73ce5908de74a62bb9267d0b1651c72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5648ea51493544a989bf87593657c2dd in term 0.
I20260806 06:31:11.096585 10002 raft_consensus.cc:2454] T 0536db0542ac4ec881d8f81b7db102cf P 618e706add47425f93508b6d2ae311cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5648ea51493544a989bf87593657c2dd in term 0.
I20260806 06:31:11.096796 10311 leader_election.cc:302] T 0536db0542ac4ec881d8f81b7db102cf P 5648ea51493544a989bf87593657c2dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5648ea51493544a989bf87593657c2dd, a73ce5908de74a62bb9267d0b1651c72; no voters:
I20260806 06:31:11.096918 10510 raft_consensus.cc:2781] T 0536db0542ac4ec881d8f81b7db102cf P 5648ea51493544a989bf87593657c2dd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:31:11.096972 10510 raft_consensus.cc:482] T 0536db0542ac4ec881d8f81b7db102cf P 5648ea51493544a989bf87593657c2dd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:31:11.096997 10510 raft_consensus.cc:3037] T 0536db0542ac4ec881d8f81b7db102cf P 5648ea51493544a989bf87593657c2dd [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:11.097630 10510 raft_consensus.cc:504] T 0536db0542ac4ec881d8f81b7db102cf P 5648ea51493544a989bf87593657c2dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } }
I20260806 06:31:11.097729 10510 leader_election.cc:288] T 0536db0542ac4ec881d8f81b7db102cf P 5648ea51493544a989bf87593657c2dd [CANDIDATE]: Term 1 election: Requested vote from peers 618e706add47425f93508b6d2ae311cd (127.7.43.65:43601), a73ce5908de74a62bb9267d0b1651c72 (127.7.43.66:34453)
I20260806 06:31:11.097896 10197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0536db0542ac4ec881d8f81b7db102cf" candidate_uuid: "5648ea51493544a989bf87593657c2dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a73ce5908de74a62bb9267d0b1651c72"
I20260806 06:31:11.097976 10197 raft_consensus.cc:3037] T 0536db0542ac4ec881d8f81b7db102cf P a73ce5908de74a62bb9267d0b1651c72 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:11.098429 10002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0536db0542ac4ec881d8f81b7db102cf" candidate_uuid: "5648ea51493544a989bf87593657c2dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "618e706add47425f93508b6d2ae311cd"
I20260806 06:31:11.098502 10002 raft_consensus.cc:3037] T 0536db0542ac4ec881d8f81b7db102cf P 618e706add47425f93508b6d2ae311cd [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:11.098802 10197 raft_consensus.cc:2454] T 0536db0542ac4ec881d8f81b7db102cf P a73ce5908de74a62bb9267d0b1651c72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5648ea51493544a989bf87593657c2dd in term 1.
I20260806 06:31:11.098965 10311 leader_election.cc:302] T 0536db0542ac4ec881d8f81b7db102cf P 5648ea51493544a989bf87593657c2dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5648ea51493544a989bf87593657c2dd, a73ce5908de74a62bb9267d0b1651c72; no voters:
I20260806 06:31:11.099042 10510 raft_consensus.cc:2781] T 0536db0542ac4ec881d8f81b7db102cf P 5648ea51493544a989bf87593657c2dd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:31:11.099182 10510 raft_consensus.cc:686] T 0536db0542ac4ec881d8f81b7db102cf P 5648ea51493544a989bf87593657c2dd [term 1 LEADER]: Becoming Leader. State: Replica: 5648ea51493544a989bf87593657c2dd, State: Running, Role: LEADER
I20260806 06:31:11.099215 10002 raft_consensus.cc:2454] T 0536db0542ac4ec881d8f81b7db102cf P 618e706add47425f93508b6d2ae311cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5648ea51493544a989bf87593657c2dd in term 1.
I20260806 06:31:11.099285 10510 consensus_queue.cc:237] T 0536db0542ac4ec881d8f81b7db102cf P 5648ea51493544a989bf87593657c2dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } }
I20260806 06:31:11.100080 9788 catalog_manager.cc:5495] T 0536db0542ac4ec881d8f81b7db102cf P 5648ea51493544a989bf87593657c2dd reported cstate change: term changed from 0 to 1, leader changed from <none> to 5648ea51493544a989bf87593657c2dd (127.7.43.67). New cstate: current_term: 1 leader_uuid: "5648ea51493544a989bf87593657c2dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } health_report { overall_health: HEALTHY } } }
Aug 06 06:31:11 dist-test-slave-6zbq krb5kdc[9725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997870, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.126@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:31:11.140981 9860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:11.134936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41112 (local address 127.7.43.126:39573)
0806 06:31:11.134997 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:11.134999 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:11.135064 (+ 65us) server_negotiation.cc:407] Connection header received
0806 06:31:11.135126 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:11.135127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:11.135137 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:11.135183 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:11.135606 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:11.136063 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:11.136701 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:11.136875 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:11.136927 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:11.138857 (+ 1930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:11.138859 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:11.138859 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:11.138866 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:11.139960 (+ 1094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:11.140281 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:11.140282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:11.140282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:11.140330 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:11.140481 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:11.140482 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:11.140482 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:11.140583 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:11.140641 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:11.140779 (+ 138us) server_negotiation.cc:299] Negotiation successful
0806 06:31:11.140863 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 06:31:11.141053 10523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:11.134848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:39573 (local address 127.0.0.1:41112)
0806 06:31:11.134998 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:11.135008 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:11.135070 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:11.135195 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:11.135195 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:11.135481 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:11.135523 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:11.135526 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:11.136131 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:11.136132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:11.136627 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:11.136629 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:11.136955 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:11.136957 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:11.137001 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:11.137444 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:11.137449 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:11.138782 (+ 1333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:11.140028 (+ 1246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:11.140035 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:11.140038 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:11.140210 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:11.140381 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:11.140382 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:11.140382 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:11.140433 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:11.140640 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:11.140647 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:11.140693 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 06:31:11.140751 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 06:31:11.140936 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":75,"thread_start_us":78,"threads_started":1}
I20260806 06:31:11.142738 9788 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:41112:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 06:31:11.142925 9788 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:31:11.144275 10173 tablet_service.cc:1467] Processing CreateTablet for tablet f9a084841a41436796db8209dd72a40f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=490687fe0b694b3080c7451166270ece]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 06:31:11.144275 9975 tablet_service.cc:1467] Processing CreateTablet for tablet f9a084841a41436796db8209dd72a40f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=490687fe0b694b3080c7451166270ece]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 06:31:11.144434 10173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9a084841a41436796db8209dd72a40f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:11.144434 9975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9a084841a41436796db8209dd72a40f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:11.144429 10371 tablet_service.cc:1467] Processing CreateTablet for tablet f9a084841a41436796db8209dd72a40f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=490687fe0b694b3080c7451166270ece]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 06:31:11.144580 10371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9a084841a41436796db8209dd72a40f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:11.145720 10499 tablet_bootstrap.cc:492] T f9a084841a41436796db8209dd72a40f P 5648ea51493544a989bf87593657c2dd: Bootstrap starting.
I20260806 06:31:11.145881 10501 tablet_bootstrap.cc:492] T f9a084841a41436796db8209dd72a40f P 618e706add47425f93508b6d2ae311cd: Bootstrap starting.
I20260806 06:31:11.145881 10505 tablet_bootstrap.cc:492] T f9a084841a41436796db8209dd72a40f P a73ce5908de74a62bb9267d0b1651c72: Bootstrap starting.
I20260806 06:31:11.146147 10499 tablet_bootstrap.cc:654] T f9a084841a41436796db8209dd72a40f P 5648ea51493544a989bf87593657c2dd: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:11.146296 10501 tablet_bootstrap.cc:654] T f9a084841a41436796db8209dd72a40f P 618e706add47425f93508b6d2ae311cd: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:11.146519 10505 tablet_bootstrap.cc:654] T f9a084841a41436796db8209dd72a40f P a73ce5908de74a62bb9267d0b1651c72: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:11.146948 10501 tablet_bootstrap.cc:492] T f9a084841a41436796db8209dd72a40f P 618e706add47425f93508b6d2ae311cd: No bootstrap required, opened a new log
I20260806 06:31:11.147001 10501 ts_tablet_manager.cc:1397] T f9a084841a41436796db8209dd72a40f P 618e706add47425f93508b6d2ae311cd: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:31:11.147045 10499 tablet_bootstrap.cc:492] T f9a084841a41436796db8209dd72a40f P 5648ea51493544a989bf87593657c2dd: No bootstrap required, opened a new log
I20260806 06:31:11.147086 10499 ts_tablet_manager.cc:1397] T f9a084841a41436796db8209dd72a40f P 5648ea51493544a989bf87593657c2dd: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 06:31:11.147142 10501 raft_consensus.cc:348] T f9a084841a41436796db8209dd72a40f P 618e706add47425f93508b6d2ae311cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } }
I20260806 06:31:11.147189 10501 raft_consensus.cc:374] T f9a084841a41436796db8209dd72a40f P 618e706add47425f93508b6d2ae311cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:11.147185 10505 tablet_bootstrap.cc:492] T f9a084841a41436796db8209dd72a40f P a73ce5908de74a62bb9267d0b1651c72: No bootstrap required, opened a new log
I20260806 06:31:11.147202 10501 raft_consensus.cc:729] T f9a084841a41436796db8209dd72a40f P 618e706add47425f93508b6d2ae311cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 618e706add47425f93508b6d2ae311cd, State: Initialized, Role: FOLLOWER
I20260806 06:31:11.147223 10505 ts_tablet_manager.cc:1397] T f9a084841a41436796db8209dd72a40f P a73ce5908de74a62bb9267d0b1651c72: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 06:31:11.147236 10501 consensus_queue.cc:260] T f9a084841a41436796db8209dd72a40f P 618e706add47425f93508b6d2ae311cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } }
I20260806 06:31:11.147236 10499 raft_consensus.cc:348] T f9a084841a41436796db8209dd72a40f P 5648ea51493544a989bf87593657c2dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } }
I20260806 06:31:11.147312 10499 raft_consensus.cc:374] T f9a084841a41436796db8209dd72a40f P 5648ea51493544a989bf87593657c2dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:11.147332 10499 raft_consensus.cc:729] T f9a084841a41436796db8209dd72a40f P 5648ea51493544a989bf87593657c2dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5648ea51493544a989bf87593657c2dd, State: Initialized, Role: FOLLOWER
I20260806 06:31:11.147326 10501 ts_tablet_manager.cc:1428] T f9a084841a41436796db8209dd72a40f P 618e706add47425f93508b6d2ae311cd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:11.147374 10499 consensus_queue.cc:260] T f9a084841a41436796db8209dd72a40f P 5648ea51493544a989bf87593657c2dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } }
I20260806 06:31:11.147378 10505 raft_consensus.cc:348] T f9a084841a41436796db8209dd72a40f P a73ce5908de74a62bb9267d0b1651c72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } }
I20260806 06:31:11.147378 10507 tablet_replica.cc:439] T f9a084841a41436796db8209dd72a40f P 618e706add47425f93508b6d2ae311cd: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } } }
I20260806 06:31:11.147451 10505 raft_consensus.cc:374] T f9a084841a41436796db8209dd72a40f P a73ce5908de74a62bb9267d0b1651c72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:11.147452 10499 ts_tablet_manager.cc:1428] T f9a084841a41436796db8209dd72a40f P 5648ea51493544a989bf87593657c2dd: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 06:31:11.147454 10507 tablet_replica.cc:442] T f9a084841a41436796db8209dd72a40f P 618e706add47425f93508b6d2ae311cd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:31:11.147475 10505 raft_consensus.cc:729] T f9a084841a41436796db8209dd72a40f P a73ce5908de74a62bb9267d0b1651c72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a73ce5908de74a62bb9267d0b1651c72, State: Initialized, Role: FOLLOWER
I20260806 06:31:11.147511 10505 consensus_queue.cc:260] T f9a084841a41436796db8209dd72a40f P a73ce5908de74a62bb9267d0b1651c72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } }
I20260806 06:31:11.147594 10505 ts_tablet_manager.cc:1428] T f9a084841a41436796db8209dd72a40f P a73ce5908de74a62bb9267d0b1651c72: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:11.147569 10510 tablet_replica.cc:439] T f9a084841a41436796db8209dd72a40f P 5648ea51493544a989bf87593657c2dd: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } } }
I20260806 06:31:11.147627 10510 tablet_replica.cc:442] T f9a084841a41436796db8209dd72a40f P 5648ea51493544a989bf87593657c2dd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:31:11.147671 10513 tablet_replica.cc:439] T f9a084841a41436796db8209dd72a40f P a73ce5908de74a62bb9267d0b1651c72: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } } }
I20260806 06:31:11.147771 10513 tablet_replica.cc:442] T f9a084841a41436796db8209dd72a40f P a73ce5908de74a62bb9267d0b1651c72: This TxnStatusTablet replica's current role is: FOLLOWER
W20260806 06:31:11.150274 10468 tablet.cc:2369] T 0536db0542ac4ec881d8f81b7db102cf P 5648ea51493544a989bf87593657c2dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:31:11.229382 10507 raft_consensus.cc:482] T f9a084841a41436796db8209dd72a40f P 618e706add47425f93508b6d2ae311cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:31:11.229457 10507 raft_consensus.cc:504] T f9a084841a41436796db8209dd72a40f P 618e706add47425f93508b6d2ae311cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } }
I20260806 06:31:11.229760 10507 leader_election.cc:288] T f9a084841a41436796db8209dd72a40f P 618e706add47425f93508b6d2ae311cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a73ce5908de74a62bb9267d0b1651c72 (127.7.43.66:34453), 5648ea51493544a989bf87593657c2dd (127.7.43.67:33315)
I20260806 06:31:11.233368 10528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:11.229944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.67:33315 (local address 127.7.43.65:41333)
0806 06:31:11.230102 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:11.230114 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:11.230184 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:11.230326 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:11.230327 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:11.230330 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:31:11.230442 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:11.230445 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:11.231322 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:11.231323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:11.232584 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:11.232588 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:11.233172 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:11.233173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:11.233220 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:11.233223 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:31:11.233270 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 06:31:11.233302 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":66,"thread_start_us":75,"threads_started":1}
I20260806 06:31:11.233379 10498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:11.230031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:41333 (local address 127.7.43.67:33315)
0806 06:31:11.230075 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:11.230077 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:11.230187 (+ 110us) server_negotiation.cc:407] Connection header received
0806 06:31:11.230242 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:11.230243 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:11.230255 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:11.230315 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:31:11.230507 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:11.231247 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:11.232662 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:11.233080 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:11.233146 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:11.233157 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:11.233281 (+ 124us) server_negotiation.cc:299] Negotiation successful
0806 06:31:11.233312 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260806 06:31:11.233736 10530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:11.229944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.66:34453 (local address 127.7.43.65:54835)
0806 06:31:11.230117 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:11.230125 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:11.230184 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:11.230326 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:11.230327 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:11.230329 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:31:11.230442 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:11.230444 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:11.231321 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:11.231321 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:11.232852 (+ 1531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:11.232854 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:11.233485 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:11.233486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:11.233553 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:11.233557 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:31:11.233641 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 06:31:11.233678 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":50,"thread_start_us":72,"threads_started":1}
I20260806 06:31:11.233758 10495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:11.230003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:54835 (local address 127.7.43.66:34453)
0806 06:31:11.230077 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:11.230078 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:11.230178 (+ 100us) server_negotiation.cc:407] Connection header received
0806 06:31:11.230247 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:11.230248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:11.230255 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:11.230315 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:31:11.230506 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:11.231243 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:11.232928 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:11.233415 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:11.233483 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:11.233493 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:11.233658 (+ 165us) server_negotiation.cc:299] Negotiation successful
0806 06:31:11.233700 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 06:31:11.233767 10404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9a084841a41436796db8209dd72a40f" candidate_uuid: "618e706add47425f93508b6d2ae311cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5648ea51493544a989bf87593657c2dd" is_pre_election: true
I20260806 06:31:11.233865 10404 raft_consensus.cc:2454] T f9a084841a41436796db8209dd72a40f P 5648ea51493544a989bf87593657c2dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 618e706add47425f93508b6d2ae311cd in term 0.
I20260806 06:31:11.234036 10197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9a084841a41436796db8209dd72a40f" candidate_uuid: "618e706add47425f93508b6d2ae311cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a73ce5908de74a62bb9267d0b1651c72" is_pre_election: true
I20260806 06:31:11.234126 10197 raft_consensus.cc:2454] T f9a084841a41436796db8209dd72a40f P a73ce5908de74a62bb9267d0b1651c72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 618e706add47425f93508b6d2ae311cd in term 0.
I20260806 06:31:11.234110 9915 leader_election.cc:302] T f9a084841a41436796db8209dd72a40f P 618e706add47425f93508b6d2ae311cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5648ea51493544a989bf87593657c2dd, 618e706add47425f93508b6d2ae311cd; no voters:
I20260806 06:31:11.234231 10507 raft_consensus.cc:2781] T f9a084841a41436796db8209dd72a40f P 618e706add47425f93508b6d2ae311cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:31:11.234270 10507 raft_consensus.cc:482] T f9a084841a41436796db8209dd72a40f P 618e706add47425f93508b6d2ae311cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:31:11.234293 10507 raft_consensus.cc:3037] T f9a084841a41436796db8209dd72a40f P 618e706add47425f93508b6d2ae311cd [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:11.234937 10507 raft_consensus.cc:504] T f9a084841a41436796db8209dd72a40f P 618e706add47425f93508b6d2ae311cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } }
I20260806 06:31:11.235054 10507 leader_election.cc:288] T f9a084841a41436796db8209dd72a40f P 618e706add47425f93508b6d2ae311cd [CANDIDATE]: Term 1 election: Requested vote from peers a73ce5908de74a62bb9267d0b1651c72 (127.7.43.66:34453), 5648ea51493544a989bf87593657c2dd (127.7.43.67:33315)
I20260806 06:31:11.235193 10404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9a084841a41436796db8209dd72a40f" candidate_uuid: "618e706add47425f93508b6d2ae311cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5648ea51493544a989bf87593657c2dd"
I20260806 06:31:11.235206 10197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9a084841a41436796db8209dd72a40f" candidate_uuid: "618e706add47425f93508b6d2ae311cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a73ce5908de74a62bb9267d0b1651c72"
I20260806 06:31:11.235255 10404 raft_consensus.cc:3037] T f9a084841a41436796db8209dd72a40f P 5648ea51493544a989bf87593657c2dd [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:11.235270 10197 raft_consensus.cc:3037] T f9a084841a41436796db8209dd72a40f P a73ce5908de74a62bb9267d0b1651c72 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:11.235863 10197 raft_consensus.cc:2454] T f9a084841a41436796db8209dd72a40f P a73ce5908de74a62bb9267d0b1651c72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 618e706add47425f93508b6d2ae311cd in term 1.
I20260806 06:31:11.235898 10404 raft_consensus.cc:2454] T f9a084841a41436796db8209dd72a40f P 5648ea51493544a989bf87593657c2dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 618e706add47425f93508b6d2ae311cd in term 1.
I20260806 06:31:11.236066 9917 leader_election.cc:302] T f9a084841a41436796db8209dd72a40f P 618e706add47425f93508b6d2ae311cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 618e706add47425f93508b6d2ae311cd, a73ce5908de74a62bb9267d0b1651c72; no voters:
I20260806 06:31:11.236174 10507 raft_consensus.cc:2781] T f9a084841a41436796db8209dd72a40f P 618e706add47425f93508b6d2ae311cd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:31:11.236292 10507 raft_consensus.cc:686] T f9a084841a41436796db8209dd72a40f P 618e706add47425f93508b6d2ae311cd [term 1 LEADER]: Becoming Leader. State: Replica: 618e706add47425f93508b6d2ae311cd, State: Running, Role: LEADER
I20260806 06:31:11.236382 10507 consensus_queue.cc:237] T f9a084841a41436796db8209dd72a40f P 618e706add47425f93508b6d2ae311cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } }
I20260806 06:31:11.236832 10531 tablet_replica.cc:439] T f9a084841a41436796db8209dd72a40f P 618e706add47425f93508b6d2ae311cd: TxnStatusTablet state changed. Reason: New leader 618e706add47425f93508b6d2ae311cd. Latest consensus state: current_term: 1 leader_uuid: "618e706add47425f93508b6d2ae311cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } } }
I20260806 06:31:11.236917 10531 tablet_replica.cc:442] T f9a084841a41436796db8209dd72a40f P 618e706add47425f93508b6d2ae311cd: This TxnStatusTablet replica's current role is: LEADER
I20260806 06:31:11.237102 10533 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 06:31:11.237067 9788 catalog_manager.cc:5495] T f9a084841a41436796db8209dd72a40f P 618e706add47425f93508b6d2ae311cd reported cstate change: term changed from 0 to 1, leader changed from <none> to 618e706add47425f93508b6d2ae311cd (127.7.43.65). New cstate: current_term: 1 leader_uuid: "618e706add47425f93508b6d2ae311cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } health_report { overall_health: UNKNOWN } } }
I20260806 06:31:11.237211 10533 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 06:31:11.263613 10523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:11.261064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:43601 (local address 127.0.0.1:40368)
0806 06:31:11.261134 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:11.261145 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:11.261186 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:11.261346 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:11.261346 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:11.261347 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 06:31:11.261387 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:11.261389 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:11.262216 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:11.262216 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:11.262797 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:11.262800 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:11.263223 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:11.263224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:11.263261 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:11.263275 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 06:31:11.263506 (+ 231us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 06:31:11.263509 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:31:11.263531 (+ 22us) client_negotiation.cc:240] Negotiation successful
0806 06:31:11.263560 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260806 06:31:11.263628 10497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:11.261117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40368 (local address 127.7.43.65:43601)
0806 06:31:11.261164 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:11.261166 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:11.261175 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:31:11.261252 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:11.261253 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:11.261272 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:11.261346 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:31:11.261434 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:11.262152 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:11.262873 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:11.263157 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:11.263232 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:11.263343 (+ 111us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:31:11.263454 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:31:11.263527 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:11.263541 (+ 14us) server_negotiation.cc:299] Negotiation successful
0806 06:31:11.263561 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 06:31:11.265211 10197 raft_consensus.cc:1261] T f9a084841a41436796db8209dd72a40f P a73ce5908de74a62bb9267d0b1651c72 [term 1 FOLLOWER]: Refusing update from remote peer 618e706add47425f93508b6d2ae311cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:31:11.265246 10404 raft_consensus.cc:1261] T f9a084841a41436796db8209dd72a40f P 5648ea51493544a989bf87593657c2dd [term 1 FOLLOWER]: Refusing update from remote peer 618e706add47425f93508b6d2ae311cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:31:11.265496 10507 consensus_queue.cc:1035] T f9a084841a41436796db8209dd72a40f P 618e706add47425f93508b6d2ae311cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:31:11.265637 10531 consensus_queue.cc:1035] T f9a084841a41436796db8209dd72a40f P 618e706add47425f93508b6d2ae311cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:31:11.266728 10510 tablet_replica.cc:439] T f9a084841a41436796db8209dd72a40f P 5648ea51493544a989bf87593657c2dd: TxnStatusTablet state changed. Reason: New leader 618e706add47425f93508b6d2ae311cd. Latest consensus state: current_term: 1 leader_uuid: "618e706add47425f93508b6d2ae311cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } } }
I20260806 06:31:11.266757 10513 tablet_replica.cc:439] T f9a084841a41436796db8209dd72a40f P a73ce5908de74a62bb9267d0b1651c72: TxnStatusTablet state changed. Reason: New leader 618e706add47425f93508b6d2ae311cd. Latest consensus state: current_term: 1 leader_uuid: "618e706add47425f93508b6d2ae311cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } } }
I20260806 06:31:11.266814 10510 tablet_replica.cc:442] T f9a084841a41436796db8209dd72a40f P 5648ea51493544a989bf87593657c2dd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:31:11.266837 10513 tablet_replica.cc:442] T f9a084841a41436796db8209dd72a40f P a73ce5908de74a62bb9267d0b1651c72: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:31:11.267441 10531 tablet_replica.cc:439] T f9a084841a41436796db8209dd72a40f P 618e706add47425f93508b6d2ae311cd: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "618e706add47425f93508b6d2ae311cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } } }
I20260806 06:31:11.267511 10531 tablet_replica.cc:442] T f9a084841a41436796db8209dd72a40f P 618e706add47425f93508b6d2ae311cd: This TxnStatusTablet replica's current role is: LEADER
I20260806 06:31:11.268188 10510 tablet_replica.cc:439] T f9a084841a41436796db8209dd72a40f P 5648ea51493544a989bf87593657c2dd: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "618e706add47425f93508b6d2ae311cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } } }
I20260806 06:31:11.268276 10510 tablet_replica.cc:442] T f9a084841a41436796db8209dd72a40f P 5648ea51493544a989bf87593657c2dd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:31:11.268275 10507 tablet_replica.cc:439] T f9a084841a41436796db8209dd72a40f P 618e706add47425f93508b6d2ae311cd: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "618e706add47425f93508b6d2ae311cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } } }
I20260806 06:31:11.268342 10507 tablet_replica.cc:442] T f9a084841a41436796db8209dd72a40f P 618e706add47425f93508b6d2ae311cd: This TxnStatusTablet replica's current role is: LEADER
I20260806 06:31:11.268385 10538 mvcc.cc:205] Tried to move back new op lower bound from 7315447280700030976 to 7315447280584724480. Current Snapshot: MvccSnapshot[applied={T|T < 7315447280700030976}]
I20260806 06:31:11.268936 10513 tablet_replica.cc:439] T f9a084841a41436796db8209dd72a40f P a73ce5908de74a62bb9267d0b1651c72: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "618e706add47425f93508b6d2ae311cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } } }
I20260806 06:31:11.269011 10513 tablet_replica.cc:442] T f9a084841a41436796db8209dd72a40f P a73ce5908de74a62bb9267d0b1651c72: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:31:11.269515 10537 mvcc.cc:205] Tried to move back new op lower bound from 7315447280700030976 to 7315447280584724480. Current Snapshot: MvccSnapshot[applied={T|T < 7315447280700030976 or (T in {7315447280700030976})}]
I20260806 06:31:11.272972 10498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:11.270120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60300 (local address 127.7.43.67:33315)
0806 06:31:11.270209 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:11.270211 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:11.270548 (+ 337us) server_negotiation.cc:407] Connection header received
0806 06:31:11.270577 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:11.270578 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:11.270593 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:11.270643 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:31:11.270816 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:11.271521 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:11.272249 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:11.272546 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:11.272612 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:11.272699 (+ 87us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:31:11.272801 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:31:11.272866 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:11.272888 (+ 22us) server_negotiation.cc:299] Negotiation successful
0806 06:31:11.272906 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 06:31:11.277418 10497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:11.275105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:51525 (local address 127.7.43.65:43601)
0806 06:31:11.275139 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:11.275141 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:11.275179 (+ 38us) server_negotiation.cc:407] Connection header received
0806 06:31:11.275253 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:11.275254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:11.275270 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:11.275310 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:31:11.275418 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:11.275850 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:11.276872 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:11.277032 (+ 160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:11.277071 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:11.277259 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:31:11.277283 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:31:11.277314 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:11.277367 (+ 53us) server_negotiation.cc:299] Negotiation successful
0806 06:31:11.277378 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 06:31:11.277473 10477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:11.275058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:43601 (local address 127.7.43.67:51525)
0806 06:31:11.275130 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:11.275141 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:11.275192 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:11.275314 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:11.275314 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:11.275315 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 06:31:11.275367 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:11.275368 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:11.275915 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:11.275916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:11.276803 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:11.276806 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:11.277085 (+ 279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:11.277086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:11.277151 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:11.277166 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 06:31:11.277318 (+ 152us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 06:31:11.277320 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:31:11.277366 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 06:31:11.277405 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 06:31:11.279627 10197 raft_consensus.cc:1261] T 0536db0542ac4ec881d8f81b7db102cf P a73ce5908de74a62bb9267d0b1651c72 [term 1 FOLLOWER]: Refusing update from remote peer 5648ea51493544a989bf87593657c2dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:31:11.279821 10002 raft_consensus.cc:1261] T 0536db0542ac4ec881d8f81b7db102cf P 618e706add47425f93508b6d2ae311cd [term 1 FOLLOWER]: Refusing update from remote peer 5648ea51493544a989bf87593657c2dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:31:11.279834 10510 consensus_queue.cc:1035] T 0536db0542ac4ec881d8f81b7db102cf P 5648ea51493544a989bf87593657c2dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:31:11.280072 10510 consensus_queue.cc:1035] T 0536db0542ac4ec881d8f81b7db102cf P 5648ea51493544a989bf87593657c2dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:31:11.282263 10536 mvcc.cc:205] Tried to move back new op lower bound from 7315447280759918592 to 7315447280023293952. Current Snapshot: MvccSnapshot[applied={T|T < 7315447280759918592}]
I20260806 06:31:11.286854 10557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:11.284505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.67:33315 (local address 127.7.43.65:40795)
0806 06:31:11.284647 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:11.284655 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:11.284705 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:11.284817 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:11.284818 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:11.284819 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 06:31:11.284876 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:11.284877 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:11.285409 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:11.285410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:11.286062 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:11.286065 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:11.286535 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:11.286536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:11.286585 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:11.286598 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 06:31:11.286743 (+ 145us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 06:31:11.286744 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 06:31:11.286773 (+ 29us) client_negotiation.cc:240] Negotiation successful
0806 06:31:11.286800 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":79,"threads_started":1}
I20260806 06:31:11.286854 10498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:11.284545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:40795 (local address 127.7.43.67:33315)
0806 06:31:11.284575 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:11.284577 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:11.284704 (+ 127us) server_negotiation.cc:407] Connection header received
0806 06:31:11.284757 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:11.284758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:11.284773 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:11.284816 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:31:11.284927 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:11.285345 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:11.286147 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:11.286465 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:11.286535 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:11.286662 (+ 127us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:31:11.286691 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:31:11.286744 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:11.286775 (+ 31us) server_negotiation.cc:299] Negotiation successful
0806 06:31:11.286792 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260806 06:31:11.293394 9790 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41102:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:31:11.293481 9790 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41102:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:31:11.294168 9790 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 50cb76ac472c4be8a76c82be91a440b9: Sending DeleteTablet for 3 replicas of tablet 0536db0542ac4ec881d8f81b7db102cf
I20260806 06:31:11.294533 10371 tablet_service.cc:1514] Processing DeleteTablet for tablet 0536db0542ac4ec881d8f81b7db102cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:31:11 UTC) from {username='kudu', principal='kudu/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:60292
I20260806 06:31:11.294564 10173 tablet_service.cc:1514] Processing DeleteTablet for tablet 0536db0542ac4ec881d8f81b7db102cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:31:11 UTC) from {username='kudu', principal='kudu/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:53890
I20260806 06:31:11.294749 9975 tablet_service.cc:1514] Processing DeleteTablet for tablet 0536db0542ac4ec881d8f81b7db102cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:31:11 UTC) from {username='kudu', principal='kudu/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:40366
I20260806 06:31:11.294956 10567 tablet_replica.cc:330] T 0536db0542ac4ec881d8f81b7db102cf P 618e706add47425f93508b6d2ae311cd: stopping tablet replica
I20260806 06:31:11.295104 10567 raft_consensus.cc:2229] T 0536db0542ac4ec881d8f81b7db102cf P 618e706add47425f93508b6d2ae311cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:31:11.295112 9790 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41102:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:31:11.295256 9790 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:31:11.295195 10567 raft_consensus.cc:2258] T 0536db0542ac4ec881d8f81b7db102cf P 618e706add47425f93508b6d2ae311cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:31:11.295791 10567 ts_tablet_manager.cc:1905] T 0536db0542ac4ec881d8f81b7db102cf P 618e706add47425f93508b6d2ae311cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:31:11.296334 10173 tablet_service.cc:1467] Processing CreateTablet for tablet bae16e66b71247ebb32bcfa2b8ecce38 (DEFAULT_TABLE table=test-table [id=826a57b5b605454a84b3a8e0cd916d24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:31:11.296448 10173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bae16e66b71247ebb32bcfa2b8ecce38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:11.297572 10567 ts_tablet_manager.cc:1918] T 0536db0542ac4ec881d8f81b7db102cf P 618e706add47425f93508b6d2ae311cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:31:11.297596 10505 tablet_bootstrap.cc:492] T bae16e66b71247ebb32bcfa2b8ecce38 P a73ce5908de74a62bb9267d0b1651c72: Bootstrap starting.
I20260806 06:31:11.297719 10567 log.cc:1196] T 0536db0542ac4ec881d8f81b7db102cf P 618e706add47425f93508b6d2ae311cd: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-0/wal/wals/0536db0542ac4ec881d8f81b7db102cf
I20260806 06:31:11.297972 10505 tablet_bootstrap.cc:654] T bae16e66b71247ebb32bcfa2b8ecce38 P a73ce5908de74a62bb9267d0b1651c72: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:11.298141 10567 ts_tablet_manager.cc:1939] T 0536db0542ac4ec881d8f81b7db102cf P 618e706add47425f93508b6d2ae311cd: Deleting consensus metadata
I20260806 06:31:11.298553 9763 catalog_manager.cc:4841] TS 618e706add47425f93508b6d2ae311cd (127.7.43.65:43601): tablet 0536db0542ac4ec881d8f81b7db102cf (table test-table [id=854097bba19f4db3b13348624195986f]) successfully deleted
I20260806 06:31:11.298686 10371 tablet_service.cc:1467] Processing CreateTablet for tablet bae16e66b71247ebb32bcfa2b8ecce38 (DEFAULT_TABLE table=test-table [id=826a57b5b605454a84b3a8e0cd916d24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:31:11.298794 10371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bae16e66b71247ebb32bcfa2b8ecce38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:11.300033 10499 tablet_bootstrap.cc:492] T bae16e66b71247ebb32bcfa2b8ecce38 P 5648ea51493544a989bf87593657c2dd: Bootstrap starting.
I20260806 06:31:11.300279 9975 tablet_service.cc:1467] Processing CreateTablet for tablet bae16e66b71247ebb32bcfa2b8ecce38 (DEFAULT_TABLE table=test-table [id=826a57b5b605454a84b3a8e0cd916d24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:31:11.300375 9975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bae16e66b71247ebb32bcfa2b8ecce38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:11.300496 10499 tablet_bootstrap.cc:654] T bae16e66b71247ebb32bcfa2b8ecce38 P 5648ea51493544a989bf87593657c2dd: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:11.301461 10501 tablet_bootstrap.cc:492] T bae16e66b71247ebb32bcfa2b8ecce38 P 618e706add47425f93508b6d2ae311cd: Bootstrap starting.
I20260806 06:31:11.301699 10565 tablet_replica.cc:330] T 0536db0542ac4ec881d8f81b7db102cf P 5648ea51493544a989bf87593657c2dd: stopping tablet replica
I20260806 06:31:11.301791 10565 raft_consensus.cc:2229] T 0536db0542ac4ec881d8f81b7db102cf P 5648ea51493544a989bf87593657c2dd [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:31:11.301867 10565 raft_consensus.cc:2258] T 0536db0542ac4ec881d8f81b7db102cf P 5648ea51493544a989bf87593657c2dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:31:11.301968 10566 tablet_replica.cc:330] T 0536db0542ac4ec881d8f81b7db102cf P a73ce5908de74a62bb9267d0b1651c72: stopping tablet replica
I20260806 06:31:11.302055 10566 raft_consensus.cc:2229] T 0536db0542ac4ec881d8f81b7db102cf P a73ce5908de74a62bb9267d0b1651c72 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:31:11.302062 10501 tablet_bootstrap.cc:654] T bae16e66b71247ebb32bcfa2b8ecce38 P 618e706add47425f93508b6d2ae311cd: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:11.302103 10566 raft_consensus.cc:2258] T 0536db0542ac4ec881d8f81b7db102cf P a73ce5908de74a62bb9267d0b1651c72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:31:11.302296 10565 ts_tablet_manager.cc:1905] T 0536db0542ac4ec881d8f81b7db102cf P 5648ea51493544a989bf87593657c2dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:31:11.302559 10505 tablet_bootstrap.cc:492] T bae16e66b71247ebb32bcfa2b8ecce38 P a73ce5908de74a62bb9267d0b1651c72: No bootstrap required, opened a new log
I20260806 06:31:11.302610 10505 ts_tablet_manager.cc:1397] T bae16e66b71247ebb32bcfa2b8ecce38 P a73ce5908de74a62bb9267d0b1651c72: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260806 06:31:11.302755 10505 raft_consensus.cc:348] T bae16e66b71247ebb32bcfa2b8ecce38 P a73ce5908de74a62bb9267d0b1651c72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } }
I20260806 06:31:11.302820 10505 raft_consensus.cc:374] T bae16e66b71247ebb32bcfa2b8ecce38 P a73ce5908de74a62bb9267d0b1651c72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:11.302839 10505 raft_consensus.cc:729] T bae16e66b71247ebb32bcfa2b8ecce38 P a73ce5908de74a62bb9267d0b1651c72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a73ce5908de74a62bb9267d0b1651c72, State: Initialized, Role: FOLLOWER
I20260806 06:31:11.302874 10505 consensus_queue.cc:260] T bae16e66b71247ebb32bcfa2b8ecce38 P a73ce5908de74a62bb9267d0b1651c72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } }
I20260806 06:31:11.302894 10501 tablet_bootstrap.cc:492] T bae16e66b71247ebb32bcfa2b8ecce38 P 618e706add47425f93508b6d2ae311cd: No bootstrap required, opened a new log
I20260806 06:31:11.302929 10501 ts_tablet_manager.cc:1397] T bae16e66b71247ebb32bcfa2b8ecce38 P 618e706add47425f93508b6d2ae311cd: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:31:11.302950 10505 ts_tablet_manager.cc:1428] T bae16e66b71247ebb32bcfa2b8ecce38 P a73ce5908de74a62bb9267d0b1651c72: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:11.303022 10501 raft_consensus.cc:348] T bae16e66b71247ebb32bcfa2b8ecce38 P 618e706add47425f93508b6d2ae311cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } }
I20260806 06:31:11.303076 10501 raft_consensus.cc:374] T bae16e66b71247ebb32bcfa2b8ecce38 P 618e706add47425f93508b6d2ae311cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:11.303092 10501 raft_consensus.cc:729] T bae16e66b71247ebb32bcfa2b8ecce38 P 618e706add47425f93508b6d2ae311cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 618e706add47425f93508b6d2ae311cd, State: Initialized, Role: FOLLOWER
I20260806 06:31:11.303125 10501 consensus_queue.cc:260] T bae16e66b71247ebb32bcfa2b8ecce38 P 618e706add47425f93508b6d2ae311cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } }
I20260806 06:31:11.303205 10501 ts_tablet_manager.cc:1428] T bae16e66b71247ebb32bcfa2b8ecce38 P 618e706add47425f93508b6d2ae311cd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:11.303632 10566 ts_tablet_manager.cc:1905] T 0536db0542ac4ec881d8f81b7db102cf P a73ce5908de74a62bb9267d0b1651c72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:31:11.304333 10565 ts_tablet_manager.cc:1918] T 0536db0542ac4ec881d8f81b7db102cf P 5648ea51493544a989bf87593657c2dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:31:11.304410 10565 log.cc:1196] T 0536db0542ac4ec881d8f81b7db102cf P 5648ea51493544a989bf87593657c2dd: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-2/wal/wals/0536db0542ac4ec881d8f81b7db102cf
I20260806 06:31:11.304653 10565 ts_tablet_manager.cc:1939] T 0536db0542ac4ec881d8f81b7db102cf P 5648ea51493544a989bf87593657c2dd: Deleting consensus metadata
I20260806 06:31:11.304877 10566 ts_tablet_manager.cc:1918] T 0536db0542ac4ec881d8f81b7db102cf P a73ce5908de74a62bb9267d0b1651c72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:31:11.304944 10566 log.cc:1196] T 0536db0542ac4ec881d8f81b7db102cf P a73ce5908de74a62bb9267d0b1651c72: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-1/wal/wals/0536db0542ac4ec881d8f81b7db102cf
I20260806 06:31:11.305032 9763 catalog_manager.cc:4841] TS 5648ea51493544a989bf87593657c2dd (127.7.43.67:33315): tablet 0536db0542ac4ec881d8f81b7db102cf (table test-table [id=854097bba19f4db3b13348624195986f]) successfully deleted
I20260806 06:31:11.305177 10566 ts_tablet_manager.cc:1939] T 0536db0542ac4ec881d8f81b7db102cf P a73ce5908de74a62bb9267d0b1651c72: Deleting consensus metadata
I20260806 06:31:11.305554 9765 catalog_manager.cc:4841] TS a73ce5908de74a62bb9267d0b1651c72 (127.7.43.66:34453): tablet 0536db0542ac4ec881d8f81b7db102cf (table test-table [id=854097bba19f4db3b13348624195986f]) successfully deleted
I20260806 06:31:11.306735 10499 tablet_bootstrap.cc:492] T bae16e66b71247ebb32bcfa2b8ecce38 P 5648ea51493544a989bf87593657c2dd: No bootstrap required, opened a new log
I20260806 06:31:11.306795 10499 ts_tablet_manager.cc:1397] T bae16e66b71247ebb32bcfa2b8ecce38 P 5648ea51493544a989bf87593657c2dd: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.001s
I20260806 06:31:11.306938 10499 raft_consensus.cc:348] T bae16e66b71247ebb32bcfa2b8ecce38 P 5648ea51493544a989bf87593657c2dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } }
I20260806 06:31:11.307011 10499 raft_consensus.cc:374] T bae16e66b71247ebb32bcfa2b8ecce38 P 5648ea51493544a989bf87593657c2dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:11.307041 10499 raft_consensus.cc:729] T bae16e66b71247ebb32bcfa2b8ecce38 P 5648ea51493544a989bf87593657c2dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5648ea51493544a989bf87593657c2dd, State: Initialized, Role: FOLLOWER
I20260806 06:31:11.307081 10499 consensus_queue.cc:260] T bae16e66b71247ebb32bcfa2b8ecce38 P 5648ea51493544a989bf87593657c2dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } }
I20260806 06:31:11.307171 10499 ts_tablet_manager.cc:1428] T bae16e66b71247ebb32bcfa2b8ecce38 P 5648ea51493544a989bf87593657c2dd: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 06:31:11.331542 10558 raft_consensus.cc:482] T bae16e66b71247ebb32bcfa2b8ecce38 P 5648ea51493544a989bf87593657c2dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:31:11.331627 10558 raft_consensus.cc:504] T bae16e66b71247ebb32bcfa2b8ecce38 P 5648ea51493544a989bf87593657c2dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } }
I20260806 06:31:11.331777 10558 leader_election.cc:288] T bae16e66b71247ebb32bcfa2b8ecce38 P 5648ea51493544a989bf87593657c2dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 618e706add47425f93508b6d2ae311cd (127.7.43.65:43601), a73ce5908de74a62bb9267d0b1651c72 (127.7.43.66:34453)
I20260806 06:31:11.331951 10002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bae16e66b71247ebb32bcfa2b8ecce38" candidate_uuid: "5648ea51493544a989bf87593657c2dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "618e706add47425f93508b6d2ae311cd" is_pre_election: true
I20260806 06:31:11.332022 10002 raft_consensus.cc:2454] T bae16e66b71247ebb32bcfa2b8ecce38 P 618e706add47425f93508b6d2ae311cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5648ea51493544a989bf87593657c2dd in term 0.
I20260806 06:31:11.331996 10197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bae16e66b71247ebb32bcfa2b8ecce38" candidate_uuid: "5648ea51493544a989bf87593657c2dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a73ce5908de74a62bb9267d0b1651c72" is_pre_election: true
I20260806 06:31:11.332082 10197 raft_consensus.cc:2454] T bae16e66b71247ebb32bcfa2b8ecce38 P a73ce5908de74a62bb9267d0b1651c72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5648ea51493544a989bf87593657c2dd in term 0.
I20260806 06:31:11.332162 10307 leader_election.cc:302] T bae16e66b71247ebb32bcfa2b8ecce38 P 5648ea51493544a989bf87593657c2dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5648ea51493544a989bf87593657c2dd, 618e706add47425f93508b6d2ae311cd; no voters:
I20260806 06:31:11.332230 10558 raft_consensus.cc:2781] T bae16e66b71247ebb32bcfa2b8ecce38 P 5648ea51493544a989bf87593657c2dd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:31:11.332253 10558 raft_consensus.cc:482] T bae16e66b71247ebb32bcfa2b8ecce38 P 5648ea51493544a989bf87593657c2dd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:31:11.332270 10558 raft_consensus.cc:3037] T bae16e66b71247ebb32bcfa2b8ecce38 P 5648ea51493544a989bf87593657c2dd [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:11.333011 10558 raft_consensus.cc:504] T bae16e66b71247ebb32bcfa2b8ecce38 P 5648ea51493544a989bf87593657c2dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } }
I20260806 06:31:11.333132 10558 leader_election.cc:288] T bae16e66b71247ebb32bcfa2b8ecce38 P 5648ea51493544a989bf87593657c2dd [CANDIDATE]: Term 1 election: Requested vote from peers 618e706add47425f93508b6d2ae311cd (127.7.43.65:43601), a73ce5908de74a62bb9267d0b1651c72 (127.7.43.66:34453)
I20260806 06:31:11.333307 10197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bae16e66b71247ebb32bcfa2b8ecce38" candidate_uuid: "5648ea51493544a989bf87593657c2dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a73ce5908de74a62bb9267d0b1651c72"
I20260806 06:31:11.333333 10002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bae16e66b71247ebb32bcfa2b8ecce38" candidate_uuid: "5648ea51493544a989bf87593657c2dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "618e706add47425f93508b6d2ae311cd"
I20260806 06:31:11.333374 10197 raft_consensus.cc:3037] T bae16e66b71247ebb32bcfa2b8ecce38 P a73ce5908de74a62bb9267d0b1651c72 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:11.333390 10002 raft_consensus.cc:3037] T bae16e66b71247ebb32bcfa2b8ecce38 P 618e706add47425f93508b6d2ae311cd [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:11.334039 10197 raft_consensus.cc:2454] T bae16e66b71247ebb32bcfa2b8ecce38 P a73ce5908de74a62bb9267d0b1651c72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5648ea51493544a989bf87593657c2dd in term 1.
I20260806 06:31:11.334061 10002 raft_consensus.cc:2454] T bae16e66b71247ebb32bcfa2b8ecce38 P 618e706add47425f93508b6d2ae311cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5648ea51493544a989bf87593657c2dd in term 1.
I20260806 06:31:11.334215 10311 leader_election.cc:302] T bae16e66b71247ebb32bcfa2b8ecce38 P 5648ea51493544a989bf87593657c2dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5648ea51493544a989bf87593657c2dd, a73ce5908de74a62bb9267d0b1651c72; no voters:
I20260806 06:31:11.334367 10558 raft_consensus.cc:2781] T bae16e66b71247ebb32bcfa2b8ecce38 P 5648ea51493544a989bf87593657c2dd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:31:11.334437 10558 raft_consensus.cc:686] T bae16e66b71247ebb32bcfa2b8ecce38 P 5648ea51493544a989bf87593657c2dd [term 1 LEADER]: Becoming Leader. State: Replica: 5648ea51493544a989bf87593657c2dd, State: Running, Role: LEADER
I20260806 06:31:11.334483 10558 consensus_queue.cc:237] T bae16e66b71247ebb32bcfa2b8ecce38 P 5648ea51493544a989bf87593657c2dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } }
I20260806 06:31:11.334945 9790 catalog_manager.cc:5495] T bae16e66b71247ebb32bcfa2b8ecce38 P 5648ea51493544a989bf87593657c2dd reported cstate change: term changed from 0 to 1, leader changed from <none> to 5648ea51493544a989bf87593657c2dd (127.7.43.67). New cstate: current_term: 1 leader_uuid: "5648ea51493544a989bf87593657c2dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } health_report { overall_health: HEALTHY } } }
I20260806 06:31:11.342633 10197 raft_consensus.cc:1261] T bae16e66b71247ebb32bcfa2b8ecce38 P a73ce5908de74a62bb9267d0b1651c72 [term 1 FOLLOWER]: Refusing update from remote peer 5648ea51493544a989bf87593657c2dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:31:11.342886 10002 raft_consensus.cc:1261] T bae16e66b71247ebb32bcfa2b8ecce38 P 618e706add47425f93508b6d2ae311cd [term 1 FOLLOWER]: Refusing update from remote peer 5648ea51493544a989bf87593657c2dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:31:11.342870 10510 consensus_queue.cc:1035] T bae16e66b71247ebb32bcfa2b8ecce38 P 5648ea51493544a989bf87593657c2dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:31:11.343056 10510 consensus_queue.cc:1035] T bae16e66b71247ebb32bcfa2b8ecce38 P 5648ea51493544a989bf87593657c2dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 06:31:11.348055 10538 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 06:31:12.866736 9788 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41102:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:31:12.866837 9788 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41102:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:31:12.867573 9788 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 50cb76ac472c4be8a76c82be91a440b9: Sending DeleteTablet for 3 replicas of tablet bae16e66b71247ebb32bcfa2b8ecce38
I20260806 06:31:12.867792 9975 tablet_service.cc:1514] Processing DeleteTablet for tablet bae16e66b71247ebb32bcfa2b8ecce38 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:31:12 UTC) from {username='kudu', principal='kudu/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:40366
I20260806 06:31:12.867806 10173 tablet_service.cc:1514] Processing DeleteTablet for tablet bae16e66b71247ebb32bcfa2b8ecce38 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:31:12 UTC) from {username='kudu', principal='kudu/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:53890
I20260806 06:31:12.868009 10610 tablet_replica.cc:330] T bae16e66b71247ebb32bcfa2b8ecce38 P a73ce5908de74a62bb9267d0b1651c72: stopping tablet replica
I20260806 06:31:12.867996 10371 tablet_service.cc:1514] Processing DeleteTablet for tablet bae16e66b71247ebb32bcfa2b8ecce38 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:31:12 UTC) from {username='kudu', principal='kudu/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:60292
I20260806 06:31:12.868068 10610 raft_consensus.cc:2229] T bae16e66b71247ebb32bcfa2b8ecce38 P a73ce5908de74a62bb9267d0b1651c72 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:31:12.868106 10610 raft_consensus.cc:2258] T bae16e66b71247ebb32bcfa2b8ecce38 P a73ce5908de74a62bb9267d0b1651c72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:31:12.868152 10611 tablet_replica.cc:330] T bae16e66b71247ebb32bcfa2b8ecce38 P 5648ea51493544a989bf87593657c2dd: stopping tablet replica
I20260806 06:31:12.868203 10611 raft_consensus.cc:2229] T bae16e66b71247ebb32bcfa2b8ecce38 P 5648ea51493544a989bf87593657c2dd [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:31:12.868254 10611 raft_consensus.cc:2258] T bae16e66b71247ebb32bcfa2b8ecce38 P 5648ea51493544a989bf87593657c2dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:31:12.870565 10611 ts_tablet_manager.cc:1905] T bae16e66b71247ebb32bcfa2b8ecce38 P 5648ea51493544a989bf87593657c2dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:31:12.871467 10609 tablet_replica.cc:330] T bae16e66b71247ebb32bcfa2b8ecce38 P 618e706add47425f93508b6d2ae311cd: stopping tablet replica
I20260806 06:31:12.871537 10609 raft_consensus.cc:2229] T bae16e66b71247ebb32bcfa2b8ecce38 P 618e706add47425f93508b6d2ae311cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:31:12.871577 10609 raft_consensus.cc:2258] T bae16e66b71247ebb32bcfa2b8ecce38 P 618e706add47425f93508b6d2ae311cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:31:12.871821 10609 ts_tablet_manager.cc:1905] T bae16e66b71247ebb32bcfa2b8ecce38 P 618e706add47425f93508b6d2ae311cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:31:12.872421 10610 ts_tablet_manager.cc:1905] T bae16e66b71247ebb32bcfa2b8ecce38 P a73ce5908de74a62bb9267d0b1651c72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:31:12.873327 10609 ts_tablet_manager.cc:1918] T bae16e66b71247ebb32bcfa2b8ecce38 P 618e706add47425f93508b6d2ae311cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 06:31:12.873391 10609 log.cc:1196] T bae16e66b71247ebb32bcfa2b8ecce38 P 618e706add47425f93508b6d2ae311cd: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-0/wal/wals/bae16e66b71247ebb32bcfa2b8ecce38
I20260806 06:31:12.873587 10609 ts_tablet_manager.cc:1939] T bae16e66b71247ebb32bcfa2b8ecce38 P 618e706add47425f93508b6d2ae311cd: Deleting consensus metadata
I20260806 06:31:12.873916 10610 ts_tablet_manager.cc:1918] T bae16e66b71247ebb32bcfa2b8ecce38 P a73ce5908de74a62bb9267d0b1651c72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 06:31:12.873904 9763 catalog_manager.cc:4841] TS 618e706add47425f93508b6d2ae311cd (127.7.43.65:43601): tablet bae16e66b71247ebb32bcfa2b8ecce38 (table test-table [id=826a57b5b605454a84b3a8e0cd916d24]) successfully deleted
I20260806 06:31:12.873973 10610 log.cc:1196] T bae16e66b71247ebb32bcfa2b8ecce38 P a73ce5908de74a62bb9267d0b1651c72: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-1/wal/wals/bae16e66b71247ebb32bcfa2b8ecce38
I20260806 06:31:12.874131 10610 ts_tablet_manager.cc:1939] T bae16e66b71247ebb32bcfa2b8ecce38 P a73ce5908de74a62bb9267d0b1651c72: Deleting consensus metadata
I20260806 06:31:12.874287 10611 ts_tablet_manager.cc:1918] T bae16e66b71247ebb32bcfa2b8ecce38 P 5648ea51493544a989bf87593657c2dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 06:31:12.874363 10611 log.cc:1196] T bae16e66b71247ebb32bcfa2b8ecce38 P 5648ea51493544a989bf87593657c2dd: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-2/wal/wals/bae16e66b71247ebb32bcfa2b8ecce38
I20260806 06:31:12.874553 10611 ts_tablet_manager.cc:1939] T bae16e66b71247ebb32bcfa2b8ecce38 P 5648ea51493544a989bf87593657c2dd: Deleting consensus metadata
I20260806 06:31:12.874577 9765 catalog_manager.cc:4841] TS a73ce5908de74a62bb9267d0b1651c72 (127.7.43.66:34453): tablet bae16e66b71247ebb32bcfa2b8ecce38 (table test-table [id=826a57b5b605454a84b3a8e0cd916d24]) successfully deleted
I20260806 06:31:12.874843 9763 catalog_manager.cc:4841] TS 5648ea51493544a989bf87593657c2dd (127.7.43.67:33315): tablet bae16e66b71247ebb32bcfa2b8ecce38 (table test-table [id=826a57b5b605454a84b3a8e0cd916d24]) successfully deleted
Aug 06 06:31:12 dist-test-slave-6zbq krb5kdc[9725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997872, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 06:31:12 dist-test-slave-6zbq krb5kdc[9725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997872, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:31:12.892109 10621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:12.885550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41114 (local address 127.7.43.126:39573)
0806 06:31:12.885687 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:12.885689 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:12.885698 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:31:12.885756 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:12.885757 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:12.885768 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:12.885822 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:12.886307 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:12.886846 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:12.887536 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:12.887739 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:12.887783 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:12.889911 (+ 2128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:12.889913 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:12.889913 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:12.889920 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:12.891328 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:12.891594 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:12.891594 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:12.891595 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:12.891641 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:12.891757 (+ 116us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:12.891757 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:12.891757 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:12.891849 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:12.891917 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:12.891984 (+ 67us) server_negotiation.cc:299] Negotiation successful
0806 06:31:12.892042 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":84,"threads_started":1}
I20260806 06:31:12.893314 9788 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41114:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:31:12.893455 9788 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:31:12.894721 9975 tablet_service.cc:1467] Processing CreateTablet for tablet 1a640589d9b742a99db4d3e2caa02fb3 (DEFAULT_TABLE table=test-table [id=1c162c1115e14b89b3f7765cd07ca1c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:31:12.894733 10173 tablet_service.cc:1467] Processing CreateTablet for tablet 1a640589d9b742a99db4d3e2caa02fb3 (DEFAULT_TABLE table=test-table [id=1c162c1115e14b89b3f7765cd07ca1c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:31:12.894819 10371 tablet_service.cc:1467] Processing CreateTablet for tablet 1a640589d9b742a99db4d3e2caa02fb3 (DEFAULT_TABLE table=test-table [id=1c162c1115e14b89b3f7765cd07ca1c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:31:12.894862 10173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a640589d9b742a99db4d3e2caa02fb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:12.894865 9975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a640589d9b742a99db4d3e2caa02fb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:12.894925 10371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a640589d9b742a99db4d3e2caa02fb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:12.895944 10623 tablet_bootstrap.cc:492] T 1a640589d9b742a99db4d3e2caa02fb3 P 618e706add47425f93508b6d2ae311cd: Bootstrap starting.
I20260806 06:31:12.896214 10625 tablet_bootstrap.cc:492] T 1a640589d9b742a99db4d3e2caa02fb3 P a73ce5908de74a62bb9267d0b1651c72: Bootstrap starting.
I20260806 06:31:12.896246 10623 tablet_bootstrap.cc:654] T 1a640589d9b742a99db4d3e2caa02fb3 P 618e706add47425f93508b6d2ae311cd: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:12.896595 10625 tablet_bootstrap.cc:654] T 1a640589d9b742a99db4d3e2caa02fb3 P a73ce5908de74a62bb9267d0b1651c72: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:12.896778 10628 tablet_bootstrap.cc:492] T 1a640589d9b742a99db4d3e2caa02fb3 P 5648ea51493544a989bf87593657c2dd: Bootstrap starting.
I20260806 06:31:12.896854 10623 tablet_bootstrap.cc:492] T 1a640589d9b742a99db4d3e2caa02fb3 P 618e706add47425f93508b6d2ae311cd: No bootstrap required, opened a new log
I20260806 06:31:12.896909 10623 ts_tablet_manager.cc:1397] T 1a640589d9b742a99db4d3e2caa02fb3 P 618e706add47425f93508b6d2ae311cd: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:31:12.897006 10623 raft_consensus.cc:348] T 1a640589d9b742a99db4d3e2caa02fb3 P 618e706add47425f93508b6d2ae311cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } }
I20260806 06:31:12.897068 10623 raft_consensus.cc:374] T 1a640589d9b742a99db4d3e2caa02fb3 P 618e706add47425f93508b6d2ae311cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:12.897080 10623 raft_consensus.cc:729] T 1a640589d9b742a99db4d3e2caa02fb3 P 618e706add47425f93508b6d2ae311cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 618e706add47425f93508b6d2ae311cd, State: Initialized, Role: FOLLOWER
I20260806 06:31:12.897115 10623 consensus_queue.cc:260] T 1a640589d9b742a99db4d3e2caa02fb3 P 618e706add47425f93508b6d2ae311cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } }
I20260806 06:31:12.897231 10623 ts_tablet_manager.cc:1428] T 1a640589d9b742a99db4d3e2caa02fb3 P 618e706add47425f93508b6d2ae311cd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:12.897423 10628 tablet_bootstrap.cc:654] T 1a640589d9b742a99db4d3e2caa02fb3 P 5648ea51493544a989bf87593657c2dd: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:12.898241 10628 tablet_bootstrap.cc:492] T 1a640589d9b742a99db4d3e2caa02fb3 P 5648ea51493544a989bf87593657c2dd: No bootstrap required, opened a new log
I20260806 06:31:12.898248 10625 tablet_bootstrap.cc:492] T 1a640589d9b742a99db4d3e2caa02fb3 P a73ce5908de74a62bb9267d0b1651c72: No bootstrap required, opened a new log
I20260806 06:31:12.898283 10625 ts_tablet_manager.cc:1397] T 1a640589d9b742a99db4d3e2caa02fb3 P a73ce5908de74a62bb9267d0b1651c72: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:31:12.898285 10628 ts_tablet_manager.cc:1397] T 1a640589d9b742a99db4d3e2caa02fb3 P 5648ea51493544a989bf87593657c2dd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:31:12.898413 10628 raft_consensus.cc:348] T 1a640589d9b742a99db4d3e2caa02fb3 P 5648ea51493544a989bf87593657c2dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } }
I20260806 06:31:12.898469 10628 raft_consensus.cc:374] T 1a640589d9b742a99db4d3e2caa02fb3 P 5648ea51493544a989bf87593657c2dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:12.898489 10628 raft_consensus.cc:729] T 1a640589d9b742a99db4d3e2caa02fb3 P 5648ea51493544a989bf87593657c2dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5648ea51493544a989bf87593657c2dd, State: Initialized, Role: FOLLOWER
I20260806 06:31:12.898447 10625 raft_consensus.cc:348] T 1a640589d9b742a99db4d3e2caa02fb3 P a73ce5908de74a62bb9267d0b1651c72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } }
I20260806 06:31:12.898514 10625 raft_consensus.cc:374] T 1a640589d9b742a99db4d3e2caa02fb3 P a73ce5908de74a62bb9267d0b1651c72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:12.898535 10625 raft_consensus.cc:729] T 1a640589d9b742a99db4d3e2caa02fb3 P a73ce5908de74a62bb9267d0b1651c72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a73ce5908de74a62bb9267d0b1651c72, State: Initialized, Role: FOLLOWER
I20260806 06:31:12.898531 10628 consensus_queue.cc:260] T 1a640589d9b742a99db4d3e2caa02fb3 P 5648ea51493544a989bf87593657c2dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } }
I20260806 06:31:12.898588 10625 consensus_queue.cc:260] T 1a640589d9b742a99db4d3e2caa02fb3 P a73ce5908de74a62bb9267d0b1651c72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } }
I20260806 06:31:12.898619 10628 ts_tablet_manager.cc:1428] T 1a640589d9b742a99db4d3e2caa02fb3 P 5648ea51493544a989bf87593657c2dd: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 06:31:12.898754 10625 ts_tablet_manager.cc:1428] T 1a640589d9b742a99db4d3e2caa02fb3 P a73ce5908de74a62bb9267d0b1651c72: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 06:31:13.023711 10507 raft_consensus.cc:482] T 1a640589d9b742a99db4d3e2caa02fb3 P 618e706add47425f93508b6d2ae311cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:31:13.023815 10507 raft_consensus.cc:504] T 1a640589d9b742a99db4d3e2caa02fb3 P 618e706add47425f93508b6d2ae311cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } }
I20260806 06:31:13.023953 10507 leader_election.cc:288] T 1a640589d9b742a99db4d3e2caa02fb3 P 618e706add47425f93508b6d2ae311cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a73ce5908de74a62bb9267d0b1651c72 (127.7.43.66:34453), 5648ea51493544a989bf87593657c2dd (127.7.43.67:33315)
I20260806 06:31:13.024160 10197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a640589d9b742a99db4d3e2caa02fb3" candidate_uuid: "618e706add47425f93508b6d2ae311cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a73ce5908de74a62bb9267d0b1651c72" is_pre_election: true
I20260806 06:31:13.024194 10404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a640589d9b742a99db4d3e2caa02fb3" candidate_uuid: "618e706add47425f93508b6d2ae311cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5648ea51493544a989bf87593657c2dd" is_pre_election: true
I20260806 06:31:13.024286 10404 raft_consensus.cc:2454] T 1a640589d9b742a99db4d3e2caa02fb3 P 5648ea51493544a989bf87593657c2dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 618e706add47425f93508b6d2ae311cd in term 0.
I20260806 06:31:13.024286 10197 raft_consensus.cc:2454] T 1a640589d9b742a99db4d3e2caa02fb3 P a73ce5908de74a62bb9267d0b1651c72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 618e706add47425f93508b6d2ae311cd in term 0.
I20260806 06:31:13.024441 9915 leader_election.cc:302] T 1a640589d9b742a99db4d3e2caa02fb3 P 618e706add47425f93508b6d2ae311cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5648ea51493544a989bf87593657c2dd, 618e706add47425f93508b6d2ae311cd; no voters:
I20260806 06:31:13.024544 10507 raft_consensus.cc:2781] T 1a640589d9b742a99db4d3e2caa02fb3 P 618e706add47425f93508b6d2ae311cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:31:13.024580 10507 raft_consensus.cc:482] T 1a640589d9b742a99db4d3e2caa02fb3 P 618e706add47425f93508b6d2ae311cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:31:13.024591 10507 raft_consensus.cc:3037] T 1a640589d9b742a99db4d3e2caa02fb3 P 618e706add47425f93508b6d2ae311cd [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:13.025240 10507 raft_consensus.cc:504] T 1a640589d9b742a99db4d3e2caa02fb3 P 618e706add47425f93508b6d2ae311cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } }
I20260806 06:31:13.025357 10507 leader_election.cc:288] T 1a640589d9b742a99db4d3e2caa02fb3 P 618e706add47425f93508b6d2ae311cd [CANDIDATE]: Term 1 election: Requested vote from peers a73ce5908de74a62bb9267d0b1651c72 (127.7.43.66:34453), 5648ea51493544a989bf87593657c2dd (127.7.43.67:33315)
I20260806 06:31:13.025501 10197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a640589d9b742a99db4d3e2caa02fb3" candidate_uuid: "618e706add47425f93508b6d2ae311cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a73ce5908de74a62bb9267d0b1651c72"
I20260806 06:31:13.025496 10404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a640589d9b742a99db4d3e2caa02fb3" candidate_uuid: "618e706add47425f93508b6d2ae311cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5648ea51493544a989bf87593657c2dd"
I20260806 06:31:13.025584 10404 raft_consensus.cc:3037] T 1a640589d9b742a99db4d3e2caa02fb3 P 5648ea51493544a989bf87593657c2dd [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:13.025583 10197 raft_consensus.cc:3037] T 1a640589d9b742a99db4d3e2caa02fb3 P a73ce5908de74a62bb9267d0b1651c72 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:13.026386 10404 raft_consensus.cc:2454] T 1a640589d9b742a99db4d3e2caa02fb3 P 5648ea51493544a989bf87593657c2dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 618e706add47425f93508b6d2ae311cd in term 1.
I20260806 06:31:13.026556 9915 leader_election.cc:302] T 1a640589d9b742a99db4d3e2caa02fb3 P 618e706add47425f93508b6d2ae311cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5648ea51493544a989bf87593657c2dd, 618e706add47425f93508b6d2ae311cd; no voters:
I20260806 06:31:13.026602 10197 raft_consensus.cc:2454] T 1a640589d9b742a99db4d3e2caa02fb3 P a73ce5908de74a62bb9267d0b1651c72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 618e706add47425f93508b6d2ae311cd in term 1.
I20260806 06:31:13.026660 10507 raft_consensus.cc:2781] T 1a640589d9b742a99db4d3e2caa02fb3 P 618e706add47425f93508b6d2ae311cd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:31:13.026721 10507 raft_consensus.cc:686] T 1a640589d9b742a99db4d3e2caa02fb3 P 618e706add47425f93508b6d2ae311cd [term 1 LEADER]: Becoming Leader. State: Replica: 618e706add47425f93508b6d2ae311cd, State: Running, Role: LEADER
I20260806 06:31:13.026774 10507 consensus_queue.cc:237] T 1a640589d9b742a99db4d3e2caa02fb3 P 618e706add47425f93508b6d2ae311cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } }
I20260806 06:31:13.027345 9788 catalog_manager.cc:5495] T 1a640589d9b742a99db4d3e2caa02fb3 P 618e706add47425f93508b6d2ae311cd reported cstate change: term changed from 0 to 1, leader changed from <none> to 618e706add47425f93508b6d2ae311cd (127.7.43.65). New cstate: current_term: 1 leader_uuid: "618e706add47425f93508b6d2ae311cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } health_report { overall_health: UNKNOWN } } }
I20260806 06:31:13.043830 10634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:13.041744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40372 (local address 127.7.43.65:43601)
0806 06:31:13.041908 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:13.041910 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:13.041917 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:31:13.041935 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:13.041936 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:13.041945 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:13.042000 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:31:13.042106 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:13.042636 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:13.043344 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:13.043526 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:13.043570 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:13.043654 (+ 84us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:31:13.043677 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:31:13.043706 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:13.043744 (+ 38us) server_negotiation.cc:299] Negotiation successful
0806 06:31:13.043759 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":99,"threads_started":1}
I20260806 06:31:13.047808 10634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:13.045812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:35603 (local address 127.7.43.65:43601)
0806 06:31:13.045857 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:13.045858 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:13.045895 (+ 37us) server_negotiation.cc:407] Connection header received
0806 06:31:13.045962 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:13.045963 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:13.045970 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:13.046009 (+ 39us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:31:13.046117 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:13.046575 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:13.047247 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:13.047425 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:13.047476 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:13.047591 (+ 115us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:31:13.047619 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:31:13.047658 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:13.047713 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 06:31:13.047732 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 06:31:13.047808 10639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:13.045740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:43601 (local address 127.7.43.65:35603)
0806 06:31:13.045841 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:13.045851 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:13.045910 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:13.046026 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:13.046026 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:13.046028 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 06:31:13.046067 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:13.046068 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:13.046645 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:13.046645 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:13.047175 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:13.047176 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:13.047485 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:13.047486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:13.047535 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:13.047538 (+ 3us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 06:31:13.047666 (+ 128us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 06:31:13.047667 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 06:31:13.047703 (+ 36us) client_negotiation.cc:240] Negotiation successful
0806 06:31:13.047745 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78,"thread_start_us":66,"threads_started":1}
I20260806 06:31:13.049469 10404 raft_consensus.cc:1261] T 1a640589d9b742a99db4d3e2caa02fb3 P 5648ea51493544a989bf87593657c2dd [term 1 FOLLOWER]: Refusing update from remote peer 618e706add47425f93508b6d2ae311cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:31:13.049504 10197 raft_consensus.cc:1261] T 1a640589d9b742a99db4d3e2caa02fb3 P a73ce5908de74a62bb9267d0b1651c72 [term 1 FOLLOWER]: Refusing update from remote peer 618e706add47425f93508b6d2ae311cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:31:13.049666 10507 consensus_queue.cc:1035] T 1a640589d9b742a99db4d3e2caa02fb3 P 618e706add47425f93508b6d2ae311cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:31:13.049777 10590 consensus_queue.cc:1035] T 1a640589d9b742a99db4d3e2caa02fb3 P 618e706add47425f93508b6d2ae311cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:31:13.059798 9788 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41114:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:31:13.059890 9788 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41114:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:31:13.060458 9788 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 50cb76ac472c4be8a76c82be91a440b9: Sending DeleteTablet for 3 replicas of tablet 1a640589d9b742a99db4d3e2caa02fb3
I20260806 06:31:13.060702 9975 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a640589d9b742a99db4d3e2caa02fb3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:31:13 UTC) from {username='kudu', principal='kudu/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:40366
I20260806 06:31:13.060693 10173 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a640589d9b742a99db4d3e2caa02fb3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:31:13 UTC) from {username='kudu', principal='kudu/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:53890
I20260806 06:31:13.060711 10371 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a640589d9b742a99db4d3e2caa02fb3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:31:13 UTC) from {username='kudu', principal='kudu/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:60292
I20260806 06:31:13.060792 10609 tablet_replica.cc:330] T 1a640589d9b742a99db4d3e2caa02fb3 P 618e706add47425f93508b6d2ae311cd: stopping tablet replica
I20260806 06:31:13.060819 10611 tablet_replica.cc:330] T 1a640589d9b742a99db4d3e2caa02fb3 P 5648ea51493544a989bf87593657c2dd: stopping tablet replica
I20260806 06:31:13.060853 10609 raft_consensus.cc:2229] T 1a640589d9b742a99db4d3e2caa02fb3 P 618e706add47425f93508b6d2ae311cd [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:31:13.060884 10611 raft_consensus.cc:2229] T 1a640589d9b742a99db4d3e2caa02fb3 P 5648ea51493544a989bf87593657c2dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:31:13.060925 10611 raft_consensus.cc:2258] T 1a640589d9b742a99db4d3e2caa02fb3 P 5648ea51493544a989bf87593657c2dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:31:13.060932 10609 raft_consensus.cc:2258] T 1a640589d9b742a99db4d3e2caa02fb3 P 618e706add47425f93508b6d2ae311cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:31:13.060989 10610 tablet_replica.cc:330] T 1a640589d9b742a99db4d3e2caa02fb3 P a73ce5908de74a62bb9267d0b1651c72: stopping tablet replica
I20260806 06:31:13.061043 10610 raft_consensus.cc:2229] T 1a640589d9b742a99db4d3e2caa02fb3 P a73ce5908de74a62bb9267d0b1651c72 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:31:13.061087 10610 raft_consensus.cc:2258] T 1a640589d9b742a99db4d3e2caa02fb3 P a73ce5908de74a62bb9267d0b1651c72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:31:13.061331 10611 ts_tablet_manager.cc:1905] T 1a640589d9b742a99db4d3e2caa02fb3 P 5648ea51493544a989bf87593657c2dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:31:13.061331 10609 ts_tablet_manager.cc:1905] T 1a640589d9b742a99db4d3e2caa02fb3 P 618e706add47425f93508b6d2ae311cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:31:13.061391 10610 ts_tablet_manager.cc:1905] T 1a640589d9b742a99db4d3e2caa02fb3 P a73ce5908de74a62bb9267d0b1651c72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:31:13.062691 10610 ts_tablet_manager.cc:1918] T 1a640589d9b742a99db4d3e2caa02fb3 P a73ce5908de74a62bb9267d0b1651c72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:31:13.062743 10610 log.cc:1196] T 1a640589d9b742a99db4d3e2caa02fb3 P a73ce5908de74a62bb9267d0b1651c72: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-1/wal/wals/1a640589d9b742a99db4d3e2caa02fb3
I20260806 06:31:13.062798 10611 ts_tablet_manager.cc:1918] T 1a640589d9b742a99db4d3e2caa02fb3 P 5648ea51493544a989bf87593657c2dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:31:13.062798 10609 ts_tablet_manager.cc:1918] T 1a640589d9b742a99db4d3e2caa02fb3 P 618e706add47425f93508b6d2ae311cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:31:13.062857 10609 log.cc:1196] T 1a640589d9b742a99db4d3e2caa02fb3 P 618e706add47425f93508b6d2ae311cd: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-0/wal/wals/1a640589d9b742a99db4d3e2caa02fb3
I20260806 06:31:13.062857 10611 log.cc:1196] T 1a640589d9b742a99db4d3e2caa02fb3 P 5648ea51493544a989bf87593657c2dd: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-2/wal/wals/1a640589d9b742a99db4d3e2caa02fb3
I20260806 06:31:13.062958 10610 ts_tablet_manager.cc:1939] T 1a640589d9b742a99db4d3e2caa02fb3 P a73ce5908de74a62bb9267d0b1651c72: Deleting consensus metadata
I20260806 06:31:13.063064 10611 ts_tablet_manager.cc:1939] T 1a640589d9b742a99db4d3e2caa02fb3 P 5648ea51493544a989bf87593657c2dd: Deleting consensus metadata
I20260806 06:31:13.063064 10609 ts_tablet_manager.cc:1939] T 1a640589d9b742a99db4d3e2caa02fb3 P 618e706add47425f93508b6d2ae311cd: Deleting consensus metadata
I20260806 06:31:13.063248 9765 catalog_manager.cc:4841] TS a73ce5908de74a62bb9267d0b1651c72 (127.7.43.66:34453): tablet 1a640589d9b742a99db4d3e2caa02fb3 (table test-table [id=1c162c1115e14b89b3f7765cd07ca1c1]) successfully deleted
I20260806 06:31:13.063350 9763 catalog_manager.cc:4841] TS 618e706add47425f93508b6d2ae311cd (127.7.43.65:43601): tablet 1a640589d9b742a99db4d3e2caa02fb3 (table test-table [id=1c162c1115e14b89b3f7765cd07ca1c1]) successfully deleted
I20260806 06:31:13.063403 9763 catalog_manager.cc:4841] TS 5648ea51493544a989bf87593657c2dd (127.7.43.67:33315): tablet 1a640589d9b742a99db4d3e2caa02fb3 (table test-table [id=1c162c1115e14b89b3f7765cd07ca1c1]) successfully deleted
I20260806 06:31:13.063374 9788 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41114:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:31:13.063516 9788 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:31:13.064531 10173 tablet_service.cc:1467] Processing CreateTablet for tablet 7b14cec9eaeb4e9aa4432683ba2bf893 (DEFAULT_TABLE table=test-table [id=171cdd4db7fc4b67b45615355afcc8a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:31:13.064531 10371 tablet_service.cc:1467] Processing CreateTablet for tablet 7b14cec9eaeb4e9aa4432683ba2bf893 (DEFAULT_TABLE table=test-table [id=171cdd4db7fc4b67b45615355afcc8a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:31:13.064535 9975 tablet_service.cc:1467] Processing CreateTablet for tablet 7b14cec9eaeb4e9aa4432683ba2bf893 (DEFAULT_TABLE table=test-table [id=171cdd4db7fc4b67b45615355afcc8a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:31:13.064635 10371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b14cec9eaeb4e9aa4432683ba2bf893. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:13.064635 10173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b14cec9eaeb4e9aa4432683ba2bf893. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:13.064636 9975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b14cec9eaeb4e9aa4432683ba2bf893. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:13.065744 10623 tablet_bootstrap.cc:492] T 7b14cec9eaeb4e9aa4432683ba2bf893 P 618e706add47425f93508b6d2ae311cd: Bootstrap starting.
I20260806 06:31:13.065810 10628 tablet_bootstrap.cc:492] T 7b14cec9eaeb4e9aa4432683ba2bf893 P 5648ea51493544a989bf87593657c2dd: Bootstrap starting.
I20260806 06:31:13.066027 10623 tablet_bootstrap.cc:654] T 7b14cec9eaeb4e9aa4432683ba2bf893 P 618e706add47425f93508b6d2ae311cd: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:13.066084 10628 tablet_bootstrap.cc:654] T 7b14cec9eaeb4e9aa4432683ba2bf893 P 5648ea51493544a989bf87593657c2dd: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:13.066318 10625 tablet_bootstrap.cc:492] T 7b14cec9eaeb4e9aa4432683ba2bf893 P a73ce5908de74a62bb9267d0b1651c72: Bootstrap starting.
I20260806 06:31:13.066748 10628 tablet_bootstrap.cc:492] T 7b14cec9eaeb4e9aa4432683ba2bf893 P 5648ea51493544a989bf87593657c2dd: No bootstrap required, opened a new log
I20260806 06:31:13.066802 10628 ts_tablet_manager.cc:1397] T 7b14cec9eaeb4e9aa4432683ba2bf893 P 5648ea51493544a989bf87593657c2dd: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:31:13.066900 10625 tablet_bootstrap.cc:654] T 7b14cec9eaeb4e9aa4432683ba2bf893 P a73ce5908de74a62bb9267d0b1651c72: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:13.066916 10628 raft_consensus.cc:348] T 7b14cec9eaeb4e9aa4432683ba2bf893 P 5648ea51493544a989bf87593657c2dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } }
I20260806 06:31:13.066990 10628 raft_consensus.cc:374] T 7b14cec9eaeb4e9aa4432683ba2bf893 P 5648ea51493544a989bf87593657c2dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:13.067016 10628 raft_consensus.cc:729] T 7b14cec9eaeb4e9aa4432683ba2bf893 P 5648ea51493544a989bf87593657c2dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5648ea51493544a989bf87593657c2dd, State: Initialized, Role: FOLLOWER
I20260806 06:31:13.067056 10628 consensus_queue.cc:260] T 7b14cec9eaeb4e9aa4432683ba2bf893 P 5648ea51493544a989bf87593657c2dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } }
I20260806 06:31:13.067157 10628 ts_tablet_manager.cc:1428] T 7b14cec9eaeb4e9aa4432683ba2bf893 P 5648ea51493544a989bf87593657c2dd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:13.067504 10623 tablet_bootstrap.cc:492] T 7b14cec9eaeb4e9aa4432683ba2bf893 P 618e706add47425f93508b6d2ae311cd: No bootstrap required, opened a new log
I20260806 06:31:13.067556 10623 ts_tablet_manager.cc:1397] T 7b14cec9eaeb4e9aa4432683ba2bf893 P 618e706add47425f93508b6d2ae311cd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:31:13.067562 10625 tablet_bootstrap.cc:492] T 7b14cec9eaeb4e9aa4432683ba2bf893 P a73ce5908de74a62bb9267d0b1651c72: No bootstrap required, opened a new log
I20260806 06:31:13.067605 10625 ts_tablet_manager.cc:1397] T 7b14cec9eaeb4e9aa4432683ba2bf893 P a73ce5908de74a62bb9267d0b1651c72: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:31:13.067688 10623 raft_consensus.cc:348] T 7b14cec9eaeb4e9aa4432683ba2bf893 P 618e706add47425f93508b6d2ae311cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } }
I20260806 06:31:13.067755 10623 raft_consensus.cc:374] T 7b14cec9eaeb4e9aa4432683ba2bf893 P 618e706add47425f93508b6d2ae311cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:13.067797 10623 raft_consensus.cc:729] T 7b14cec9eaeb4e9aa4432683ba2bf893 P 618e706add47425f93508b6d2ae311cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 618e706add47425f93508b6d2ae311cd, State: Initialized, Role: FOLLOWER
I20260806 06:31:13.067832 10623 consensus_queue.cc:260] T 7b14cec9eaeb4e9aa4432683ba2bf893 P 618e706add47425f93508b6d2ae311cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } }
I20260806 06:31:13.067857 10625 raft_consensus.cc:348] T 7b14cec9eaeb4e9aa4432683ba2bf893 P a73ce5908de74a62bb9267d0b1651c72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } }
I20260806 06:31:13.067917 10623 ts_tablet_manager.cc:1428] T 7b14cec9eaeb4e9aa4432683ba2bf893 P 618e706add47425f93508b6d2ae311cd: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 06:31:13.067934 10625 raft_consensus.cc:374] T 7b14cec9eaeb4e9aa4432683ba2bf893 P a73ce5908de74a62bb9267d0b1651c72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:13.067958 10625 raft_consensus.cc:729] T 7b14cec9eaeb4e9aa4432683ba2bf893 P a73ce5908de74a62bb9267d0b1651c72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a73ce5908de74a62bb9267d0b1651c72, State: Initialized, Role: FOLLOWER
I20260806 06:31:13.067999 10625 consensus_queue.cc:260] T 7b14cec9eaeb4e9aa4432683ba2bf893 P a73ce5908de74a62bb9267d0b1651c72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } }
I20260806 06:31:13.068073 10625 ts_tablet_manager.cc:1428] T 7b14cec9eaeb4e9aa4432683ba2bf893 P a73ce5908de74a62bb9267d0b1651c72: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:13.370966 10510 raft_consensus.cc:482] T 7b14cec9eaeb4e9aa4432683ba2bf893 P 5648ea51493544a989bf87593657c2dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:31:13.371068 10510 raft_consensus.cc:504] T 7b14cec9eaeb4e9aa4432683ba2bf893 P 5648ea51493544a989bf87593657c2dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } }
I20260806 06:31:13.371254 10510 leader_election.cc:288] T 7b14cec9eaeb4e9aa4432683ba2bf893 P 5648ea51493544a989bf87593657c2dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 618e706add47425f93508b6d2ae311cd (127.7.43.65:43601), a73ce5908de74a62bb9267d0b1651c72 (127.7.43.66:34453)
I20260806 06:31:13.371465 10197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b14cec9eaeb4e9aa4432683ba2bf893" candidate_uuid: "5648ea51493544a989bf87593657c2dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a73ce5908de74a62bb9267d0b1651c72" is_pre_election: true
I20260806 06:31:13.371476 10002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b14cec9eaeb4e9aa4432683ba2bf893" candidate_uuid: "5648ea51493544a989bf87593657c2dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "618e706add47425f93508b6d2ae311cd" is_pre_election: true
I20260806 06:31:13.371553 10197 raft_consensus.cc:2454] T 7b14cec9eaeb4e9aa4432683ba2bf893 P a73ce5908de74a62bb9267d0b1651c72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5648ea51493544a989bf87593657c2dd in term 0.
I20260806 06:31:13.371573 10002 raft_consensus.cc:2454] T 7b14cec9eaeb4e9aa4432683ba2bf893 P 618e706add47425f93508b6d2ae311cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5648ea51493544a989bf87593657c2dd in term 0.
I20260806 06:31:13.371708 10311 leader_election.cc:302] T 7b14cec9eaeb4e9aa4432683ba2bf893 P 5648ea51493544a989bf87593657c2dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5648ea51493544a989bf87593657c2dd, a73ce5908de74a62bb9267d0b1651c72; no voters:
I20260806 06:31:13.371827 10510 raft_consensus.cc:2781] T 7b14cec9eaeb4e9aa4432683ba2bf893 P 5648ea51493544a989bf87593657c2dd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:31:13.371876 10510 raft_consensus.cc:482] T 7b14cec9eaeb4e9aa4432683ba2bf893 P 5648ea51493544a989bf87593657c2dd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:31:13.371893 10510 raft_consensus.cc:3037] T 7b14cec9eaeb4e9aa4432683ba2bf893 P 5648ea51493544a989bf87593657c2dd [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:13.372622 10510 raft_consensus.cc:504] T 7b14cec9eaeb4e9aa4432683ba2bf893 P 5648ea51493544a989bf87593657c2dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } }
I20260806 06:31:13.372735 10510 leader_election.cc:288] T 7b14cec9eaeb4e9aa4432683ba2bf893 P 5648ea51493544a989bf87593657c2dd [CANDIDATE]: Term 1 election: Requested vote from peers 618e706add47425f93508b6d2ae311cd (127.7.43.65:43601), a73ce5908de74a62bb9267d0b1651c72 (127.7.43.66:34453)
I20260806 06:31:13.372874 10197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b14cec9eaeb4e9aa4432683ba2bf893" candidate_uuid: "5648ea51493544a989bf87593657c2dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a73ce5908de74a62bb9267d0b1651c72"
I20260806 06:31:13.372880 10002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b14cec9eaeb4e9aa4432683ba2bf893" candidate_uuid: "5648ea51493544a989bf87593657c2dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "618e706add47425f93508b6d2ae311cd"
I20260806 06:31:13.372943 10197 raft_consensus.cc:3037] T 7b14cec9eaeb4e9aa4432683ba2bf893 P a73ce5908de74a62bb9267d0b1651c72 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:13.372944 10002 raft_consensus.cc:3037] T 7b14cec9eaeb4e9aa4432683ba2bf893 P 618e706add47425f93508b6d2ae311cd [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:13.373526 10197 raft_consensus.cc:2454] T 7b14cec9eaeb4e9aa4432683ba2bf893 P a73ce5908de74a62bb9267d0b1651c72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5648ea51493544a989bf87593657c2dd in term 1.
I20260806 06:31:13.373554 10002 raft_consensus.cc:2454] T 7b14cec9eaeb4e9aa4432683ba2bf893 P 618e706add47425f93508b6d2ae311cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5648ea51493544a989bf87593657c2dd in term 1.
I20260806 06:31:13.373695 10311 leader_election.cc:302] T 7b14cec9eaeb4e9aa4432683ba2bf893 P 5648ea51493544a989bf87593657c2dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5648ea51493544a989bf87593657c2dd, a73ce5908de74a62bb9267d0b1651c72; no voters:
I20260806 06:31:13.373803 10510 raft_consensus.cc:2781] T 7b14cec9eaeb4e9aa4432683ba2bf893 P 5648ea51493544a989bf87593657c2dd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:31:13.373910 10510 raft_consensus.cc:686] T 7b14cec9eaeb4e9aa4432683ba2bf893 P 5648ea51493544a989bf87593657c2dd [term 1 LEADER]: Becoming Leader. State: Replica: 5648ea51493544a989bf87593657c2dd, State: Running, Role: LEADER
I20260806 06:31:13.373965 10510 consensus_queue.cc:237] T 7b14cec9eaeb4e9aa4432683ba2bf893 P 5648ea51493544a989bf87593657c2dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } }
I20260806 06:31:13.374536 9788 catalog_manager.cc:5495] T 7b14cec9eaeb4e9aa4432683ba2bf893 P 5648ea51493544a989bf87593657c2dd reported cstate change: term changed from 0 to 1, leader changed from <none> to 5648ea51493544a989bf87593657c2dd (127.7.43.67). New cstate: current_term: 1 leader_uuid: "5648ea51493544a989bf87593657c2dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5648ea51493544a989bf87593657c2dd" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 33315 } health_report { overall_health: HEALTHY } } }
I20260806 06:31:13.423564 10653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:13.421590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60314 (local address 127.7.43.67:33315)
0806 06:31:13.421697 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:13.421699 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:13.421708 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:31:13.421731 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:13.421731 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:13.421741 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:13.421807 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:31:13.421897 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:13.422416 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:13.423066 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:13.423256 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:13.423298 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:13.423390 (+ 92us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:31:13.423414 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:31:13.423445 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:13.423475 (+ 30us) server_negotiation.cc:299] Negotiation successful
0806 06:31:13.423490 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":54,"threads_started":1}
I20260806 06:31:13.425835 10197 raft_consensus.cc:1261] T 7b14cec9eaeb4e9aa4432683ba2bf893 P a73ce5908de74a62bb9267d0b1651c72 [term 1 FOLLOWER]: Refusing update from remote peer 5648ea51493544a989bf87593657c2dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:31:13.425980 10002 raft_consensus.cc:1261] T 7b14cec9eaeb4e9aa4432683ba2bf893 P 618e706add47425f93508b6d2ae311cd [term 1 FOLLOWER]: Refusing update from remote peer 5648ea51493544a989bf87593657c2dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:31:13.426066 10651 consensus_queue.cc:1035] T 7b14cec9eaeb4e9aa4432683ba2bf893 P 5648ea51493544a989bf87593657c2dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "a73ce5908de74a62bb9267d0b1651c72" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 34453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:31:13.426209 10651 consensus_queue.cc:1035] T 7b14cec9eaeb4e9aa4432683ba2bf893 P 5648ea51493544a989bf87593657c2dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "618e706add47425f93508b6d2ae311cd" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 06:31:13.430365 10580 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 06:31:14.946026 9790 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41114:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:31:14.946147 9790 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41114:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:31:14.947300 9790 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 50cb76ac472c4be8a76c82be91a440b9: Sending DeleteTablet for 3 replicas of tablet 7b14cec9eaeb4e9aa4432683ba2bf893
I20260806 06:31:14.947575 9975 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b14cec9eaeb4e9aa4432683ba2bf893 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:31:14 UTC) from {username='kudu', principal='kudu/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:40366
I20260806 06:31:14.947605 10173 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b14cec9eaeb4e9aa4432683ba2bf893 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:31:14 UTC) from {username='kudu', principal='kudu/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:53890
I20260806 06:31:14.947882 10695 tablet_replica.cc:330] T 7b14cec9eaeb4e9aa4432683ba2bf893 P a73ce5908de74a62bb9267d0b1651c72: stopping tablet replica
I20260806 06:31:14.947945 10371 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b14cec9eaeb4e9aa4432683ba2bf893 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:31:14 UTC) from {username='kudu', principal='kudu/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:60292
I20260806 06:31:14.947981 10695 raft_consensus.cc:2229] T 7b14cec9eaeb4e9aa4432683ba2bf893 P a73ce5908de74a62bb9267d0b1651c72 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:31:14.948022 10695 raft_consensus.cc:2258] T 7b14cec9eaeb4e9aa4432683ba2bf893 P a73ce5908de74a62bb9267d0b1651c72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:31:14.948132 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 9892
I20260806 06:31:14.950477 10695 ts_tablet_manager.cc:1905] T 7b14cec9eaeb4e9aa4432683ba2bf893 P a73ce5908de74a62bb9267d0b1651c72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:31:14.951921 10695 ts_tablet_manager.cc:1918] T 7b14cec9eaeb4e9aa4432683ba2bf893 P a73ce5908de74a62bb9267d0b1651c72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 06:31:14.951972 10695 log.cc:1196] T 7b14cec9eaeb4e9aa4432683ba2bf893 P a73ce5908de74a62bb9267d0b1651c72: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-1/wal/wals/7b14cec9eaeb4e9aa4432683ba2bf893
I20260806 06:31:14.952185 10695 ts_tablet_manager.cc:1939] T 7b14cec9eaeb4e9aa4432683ba2bf893 P a73ce5908de74a62bb9267d0b1651c72: Deleting consensus metadata
I20260806 06:31:14.952461 9765 catalog_manager.cc:4841] TS a73ce5908de74a62bb9267d0b1651c72 (127.7.43.66:34453): tablet 7b14cec9eaeb4e9aa4432683ba2bf893 (table test-table [id=171cdd4db7fc4b67b45615355afcc8a8]) successfully deleted
I20260806 06:31:14.953377 10696 tablet_replica.cc:330] T 7b14cec9eaeb4e9aa4432683ba2bf893 P 5648ea51493544a989bf87593657c2dd: stopping tablet replica
I20260806 06:31:14.953460 10696 raft_consensus.cc:2229] T 7b14cec9eaeb4e9aa4432683ba2bf893 P 5648ea51493544a989bf87593657c2dd [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:31:14.953537 10696 raft_consensus.cc:2258] T 7b14cec9eaeb4e9aa4432683ba2bf893 P 5648ea51493544a989bf87593657c2dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:31:14.953769 10696 ts_tablet_manager.cc:1905] T 7b14cec9eaeb4e9aa4432683ba2bf893 P 5648ea51493544a989bf87593657c2dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:31:14.955120 10696 ts_tablet_manager.cc:1918] T 7b14cec9eaeb4e9aa4432683ba2bf893 P 5648ea51493544a989bf87593657c2dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 06:31:14.955189 10696 log.cc:1196] T 7b14cec9eaeb4e9aa4432683ba2bf893 P 5648ea51493544a989bf87593657c2dd: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785997861828271-7341-0/minicluster-data/ts-2/wal/wals/7b14cec9eaeb4e9aa4432683ba2bf893
I20260806 06:31:14.955382 10696 ts_tablet_manager.cc:1939] T 7b14cec9eaeb4e9aa4432683ba2bf893 P 5648ea51493544a989bf87593657c2dd: Deleting consensus metadata
I20260806 06:31:14.957540 9763 catalog_manager.cc:4841] TS 5648ea51493544a989bf87593657c2dd (127.7.43.67:33315): tablet 7b14cec9eaeb4e9aa4432683ba2bf893 (table test-table [id=171cdd4db7fc4b67b45615355afcc8a8]) successfully deleted
W20260806 06:31:14.962718 9763 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.43.65:43601 (error 108)
I20260806 06:31:14.963243 10698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:14.963059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:43601 (local address 127.0.0.1:40384)
0806 06:31:14.963173 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:14.963211 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.65:43601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":66,"threads_started":1}
I20260806 06:31:14.963285 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 10082
W20260806 06:31:14.963356 9763 catalog_manager.cc:4584] TS 618e706add47425f93508b6d2ae311cd (127.7.43.65:43601): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7b14cec9eaeb4e9aa4432683ba2bf893: Network error: Client connection negotiation failed: client connection to 127.7.43.65:43601: connect: Connection refused (error 111)
I20260806 06:31:14.972608 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 10279
I20260806 06:31:14.983372 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 9743
2026-08-06T06:31:14Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5927 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:31:15 dist-test-slave-6zbq krb5kdc[10701](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:31:15 dist-test-slave-6zbq krb5kdc[10701](info): set up 2 sockets
Aug 06 06:31:15 dist-test-slave-6zbq krb5kdc[10701](info): commencing operation
krb5kdc: starting...
W20260806 06:31:16.105501 7341 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Aug 06 06:31:16 dist-test-slave-6zbq krb5kdc[10701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:31:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:31:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.43.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
I20260806 06:31:16.212577 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:44353
--webserver_interface=127.7.43.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:34365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.43.126:44353
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:31:16.291419 10716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:16.291661 10716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:16.291693 10716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:16.292797 10716 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:31:16.292836 10716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:16.292855 10716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:16.292871 10716 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:31:16.292887 10716 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:31:16.294327 10716 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:34365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.43.126:44353
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:44353
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.43.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10716
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:16.294571 10716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:16.294759 10716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:16.299075 10726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:16.299078 10722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:16.299098 10724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:16.299230 10716 server_base.cc:1034] running on GCE node
I20260806 06:31:16.299453 10716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:16.299683 10716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:16.300817 10716 hybrid_clock.cc:648] HybridClock initialized: now 1785997876300803 us; error 32 us; skew 500 ppm
Aug 06 06:31:16 dist-test-slave-6zbq krb5kdc[10701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997876, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:16.303045 10716 init.cc:377] Logged in from keytab as kudu/127.7.43.126@KRBTEST.COM (short username kudu)
I20260806 06:31:16.303439 10716 webserver.cc:460] Webserver started at http://127.7.43.126:45905/ using document root <none> and password file <none>
I20260806 06:31:16.303638 10716 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:16.303679 10716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:16.303794 10716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:16.304718 10716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/master-0/data/instance:
uuid: "96d30f2f6de340c2bdd5380f02a9bd75"
format_stamp: "Formatted at 2026-08-06 06:31:16 on dist-test-slave-6zbq"
server_key: "2788b78750593ecb7bd82a15ae7abe46"
server_key_iv: "5a5b2ce4d089700bbbbf0aa3f3f6cc04"
server_key_version: "encryptionkey@0"
I20260806 06:31:16.305027 10716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/master-0/wal/instance:
uuid: "96d30f2f6de340c2bdd5380f02a9bd75"
format_stamp: "Formatted at 2026-08-06 06:31:16 on dist-test-slave-6zbq"
server_key: "2788b78750593ecb7bd82a15ae7abe46"
server_key_iv: "5a5b2ce4d089700bbbbf0aa3f3f6cc04"
server_key_version: "encryptionkey@0"
I20260806 06:31:16.306290 10716 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:31:16.307052 10735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:16.307219 10716 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:31:16.307303 10716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "96d30f2f6de340c2bdd5380f02a9bd75"
format_stamp: "Formatted at 2026-08-06 06:31:16 on dist-test-slave-6zbq"
server_key: "2788b78750593ecb7bd82a15ae7abe46"
server_key_iv: "5a5b2ce4d089700bbbbf0aa3f3f6cc04"
server_key_version: "encryptionkey@0"
I20260806 06:31:16.307360 10716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:16.320386 10716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:16.321916 10716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:16.322044 10716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:16.325573 10716 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.126:44353
I20260806 06:31:16.325618 10813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.126:44353 every 8 connection(s)
I20260806 06:31:16.325886 10716 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:31:16.326380 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 10716
I20260806 06:31:16.326476 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/master-0/wal/instance
I20260806 06:31:16.326663 7341 external_mini_cluster.cc:1428] Setting key 0da29dad7a7314e151f2003f8450946c
I20260806 06:31:16.326865 10814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:16.329677 10814 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96d30f2f6de340c2bdd5380f02a9bd75: Bootstrap starting.
I20260806 06:31:16.330331 10814 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 96d30f2f6de340c2bdd5380f02a9bd75: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:16.330648 10814 log.cc:824] T 00000000000000000000000000000000 P 96d30f2f6de340c2bdd5380f02a9bd75: Log is configured to *not* fsync() on all Append() calls
I20260806 06:31:16.331256 10814 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96d30f2f6de340c2bdd5380f02a9bd75: No bootstrap required, opened a new log
Aug 06 06:31:16 dist-test-slave-6zbq krb5kdc[10701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:31:16.333114 10814 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96d30f2f6de340c2bdd5380f02a9bd75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96d30f2f6de340c2bdd5380f02a9bd75" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 44353 } }
I20260806 06:31:16.333274 10814 raft_consensus.cc:374] T 00000000000000000000000000000000 P 96d30f2f6de340c2bdd5380f02a9bd75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:16.333303 10814 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96d30f2f6de340c2bdd5380f02a9bd75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96d30f2f6de340c2bdd5380f02a9bd75, State: Initialized, Role: FOLLOWER
I20260806 06:31:16.333420 10814 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96d30f2f6de340c2bdd5380f02a9bd75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96d30f2f6de340c2bdd5380f02a9bd75" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 44353 } }
I20260806 06:31:16.333484 10814 raft_consensus.cc:388] T 00000000000000000000000000000000 P 96d30f2f6de340c2bdd5380f02a9bd75 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:31:16.333524 10814 raft_consensus.cc:482] T 00000000000000000000000000000000 P 96d30f2f6de340c2bdd5380f02a9bd75 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:31:16.333570 10814 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96d30f2f6de340c2bdd5380f02a9bd75 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:16.334131 10814 raft_consensus.cc:504] T 00000000000000000000000000000000 P 96d30f2f6de340c2bdd5380f02a9bd75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96d30f2f6de340c2bdd5380f02a9bd75" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 44353 } }
I20260806 06:31:16.334228 10814 leader_election.cc:302] T 00000000000000000000000000000000 P 96d30f2f6de340c2bdd5380f02a9bd75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 96d30f2f6de340c2bdd5380f02a9bd75; no voters:
I20260806 06:31:16.334394 10814 leader_election.cc:288] T 00000000000000000000000000000000 P 96d30f2f6de340c2bdd5380f02a9bd75 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:31:16.334425 10819 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 96d30f2f6de340c2bdd5380f02a9bd75 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:31:16.334578 10819 raft_consensus.cc:686] T 00000000000000000000000000000000 P 96d30f2f6de340c2bdd5380f02a9bd75 [term 1 LEADER]: Becoming Leader. State: Replica: 96d30f2f6de340c2bdd5380f02a9bd75, State: Running, Role: LEADER
I20260806 06:31:16.334640 10814 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96d30f2f6de340c2bdd5380f02a9bd75 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:31:16.334707 10819 consensus_queue.cc:237] T 00000000000000000000000000000000 P 96d30f2f6de340c2bdd5380f02a9bd75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96d30f2f6de340c2bdd5380f02a9bd75" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 44353 } }
I20260806 06:31:16.335000 10819 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96d30f2f6de340c2bdd5380f02a9bd75 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96d30f2f6de340c2bdd5380f02a9bd75. Latest consensus state: current_term: 1 leader_uuid: "96d30f2f6de340c2bdd5380f02a9bd75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96d30f2f6de340c2bdd5380f02a9bd75" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 44353 } } }
I20260806 06:31:16.335076 10819 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96d30f2f6de340c2bdd5380f02a9bd75 [sys.catalog]: This master's current role is: LEADER
I20260806 06:31:16.335515 10821 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96d30f2f6de340c2bdd5380f02a9bd75 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "96d30f2f6de340c2bdd5380f02a9bd75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96d30f2f6de340c2bdd5380f02a9bd75" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 44353 } } }
I20260806 06:31:16.335618 10821 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96d30f2f6de340c2bdd5380f02a9bd75 [sys.catalog]: This master's current role is: LEADER
I20260806 06:31:16.336622 10817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:16.327096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54264 (local address 127.7.43.126:44353)
0806 06:31:16.327259 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:16.327268 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:16.327293 (+ 25us) server_negotiation.cc:407] Connection header received
0806 06:31:16.327823 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:16.327829 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:16.327871 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:16.327974 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:16.328710 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:16.329532 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:16.330544 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:16.330949 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:16.331045 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:16.333231 (+ 2186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:16.333235 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:16.333236 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:16.333245 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:16.334984 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:16.335411 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:16.335413 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:16.335413 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:16.335509 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:16.335730 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:16.335731 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:16.335731 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:16.336002 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:16.336162 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:16.336238 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 06:31:16.336397 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":71,"threads_started":1}
W20260806 06:31:16.336666 10839 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 96d30f2f6de340c2bdd5380f02a9bd75: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:31:16.336710 10839 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:31:16.336767 10830 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:31:16.336958 10830 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:31:16.338313 10830 catalog_manager.cc:1347] Generated new cluster ID: 21cd7c0947d241699d22a1a1e5a586ff
I20260806 06:31:16.338413 10830 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:31:16.342319 10830 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:31:16.342800 10830 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:31:16.349597 10830 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 96d30f2f6de340c2bdd5380f02a9bd75: Generated new TSK 0
I20260806 06:31:16.349741 10830 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.43.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab.
Entry for principal kudu/127.7.43.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab.
I20260806 06:31:16.373353 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.65:0
--local_ip_for_outbound_sockets=127.7.43.65
--webserver_interface=127.7.43.65
--webserver_port=0
--tserver_master_addrs=127.7.43.126:44353
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab
--principal=kudu/127.7.43.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:34365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:31:16.452823 10845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:16.453001 10845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:16.453022 10845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:16.454159 10845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:16.454200 10845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:16.454224 10845 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.65
I20260806 06:31:16.455624 10845 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:34365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab
--principal=kudu/127.7.43.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.43.65
--webserver_port=0
--tserver_master_addrs=127.7.43.126:44353
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10845
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.65
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:16.455826 10845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:16.456017 10845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:16.460530 10853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:16.460556 10852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:16.460626 10845 server_base.cc:1034] running on GCE node
W20260806 06:31:16.460578 10855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:16.460911 10845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:16.461138 10845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:16.462260 10845 hybrid_clock.cc:648] HybridClock initialized: now 1785997876462251 us; error 28 us; skew 500 ppm
Aug 06 06:31:16 dist-test-slave-6zbq krb5kdc[10701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997876, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:16.464480 10845 init.cc:377] Logged in from keytab as kudu/127.7.43.65@KRBTEST.COM (short username kudu)
I20260806 06:31:16.464838 10845 webserver.cc:460] Webserver started at http://127.7.43.65:40995/ using document root <none> and password file <none>
I20260806 06:31:16.465000 10845 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:16.465036 10845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:16.465130 10845 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:16.466018 10845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-0/data/instance:
uuid: "941af6204a3945a483c8870fcc7c9265"
format_stamp: "Formatted at 2026-08-06 06:31:16 on dist-test-slave-6zbq"
server_key: "86c6358137943989948016e6b5fcd41b"
server_key_iv: "e0b76f4d9c34e780b6f82842d60d518b"
server_key_version: "encryptionkey@0"
I20260806 06:31:16.466302 10845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-0/wal/instance:
uuid: "941af6204a3945a483c8870fcc7c9265"
format_stamp: "Formatted at 2026-08-06 06:31:16 on dist-test-slave-6zbq"
server_key: "86c6358137943989948016e6b5fcd41b"
server_key_iv: "e0b76f4d9c34e780b6f82842d60d518b"
server_key_version: "encryptionkey@0"
I20260806 06:31:16.467464 10845 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:31:16.468060 10865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:16.468209 10845 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:16.468288 10845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-0/wal
uuid: "941af6204a3945a483c8870fcc7c9265"
format_stamp: "Formatted at 2026-08-06 06:31:16 on dist-test-slave-6zbq"
server_key: "86c6358137943989948016e6b5fcd41b"
server_key_iv: "e0b76f4d9c34e780b6f82842d60d518b"
server_key_version: "encryptionkey@0"
I20260806 06:31:16.468341 10845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:16.475342 10845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:16.477190 10845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:16.477286 10845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:16.477447 10845 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:31:16.477669 10845 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:31:16.477696 10845 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:16.477728 10845 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:31:16.477746 10845 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:16.483093 10845 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.65:37365
I20260806 06:31:16.483158 11022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.65:37365 every 8 connection(s)
I20260806 06:31:16.483402 10845 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
I20260806 06:31:16.487417 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 10845
I20260806 06:31:16.487502 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-0/wal/instance
I20260806 06:31:16.487679 7341 external_mini_cluster.cc:1428] Setting key acec1fab1dbe13a3beaa3ccc9fd6fe31
Aug 06 06:31:16 dist-test-slave-6zbq krb5kdc[10701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997876, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.65@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:31:16.491918 10817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:16.484038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:34165 (local address 127.7.43.126:44353)
0806 06:31:16.484095 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:16.484097 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:16.484281 (+ 184us) server_negotiation.cc:407] Connection header received
0806 06:31:16.484681 (+ 400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:16.484682 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:16.484692 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:16.484732 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:16.485522 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:16.485994 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:16.486920 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:16.487111 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:16.487158 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:16.489312 (+ 2154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:16.489313 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:16.489314 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:16.489320 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:16.490751 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:16.491104 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:16.491105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:16.491105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:16.491168 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:16.491366 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:16.491367 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:16.491367 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:16.491490 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:16.491549 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:16.491752 (+ 203us) server_negotiation.cc:299] Negotiation successful
0806 06:31:16.491832 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 06:31:16.492031 11026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:16.484015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:44353 (local address 127.7.43.65:34165)
0806 06:31:16.484183 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:16.484213 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:16.484599 (+ 386us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:16.484791 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:16.484794 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:16.485291 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:16.485458 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:16.485462 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:16.486051 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:16.486052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:16.486860 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:16.486862 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:16.487168 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:16.487168 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:16.487257 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:16.487688 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:16.487696 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:16.489224 (+ 1528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:16.490825 (+ 1601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:16.490831 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:16.490833 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:16.491034 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:16.491228 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:16.491229 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:16.491229 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:16.491314 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:16.491547 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:16.491550 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:16.491646 (+ 96us) client_negotiation.cc:769] Sending connection context
0806 06:31:16.491715 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 06:31:16.491845 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":92,"threads_started":1}
I20260806 06:31:16.492442 11024 heartbeater.cc:347] Connected to a master server at 127.7.43.126:44353
I20260806 06:31:16.492548 11024 heartbeater.cc:464] Registering TS with master...
I20260806 06:31:16.492736 11024 heartbeater.cc:511] Master 127.7.43.126:44353 requested a full tablet report, sending...
I20260806 06:31:16.493285 10759 ts_manager.cc:194] Registered new tserver with Master: 941af6204a3945a483c8870fcc7c9265 (127.7.43.65:37365)
I20260806 06:31:16.494140 10759 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.65@KRBTEST.COM'} at 127.7.43.65:34165
WARNING: no policy specified for kudu/127.7.43.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab.
Entry for principal kudu/127.7.43.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab.
I20260806 06:31:16.510255 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.66:0
--local_ip_for_outbound_sockets=127.7.43.66
--webserver_interface=127.7.43.66
--webserver_port=0
--tserver_master_addrs=127.7.43.126:44353
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab
--principal=kudu/127.7.43.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:34365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:31:16.587464 11032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:16.587636 11032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:16.587658 11032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:16.588718 11032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:16.588749 11032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:16.588773 11032 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.66
I20260806 06:31:16.590083 11032 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:34365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab
--principal=kudu/127.7.43.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.43.66
--webserver_port=0
--tserver_master_addrs=127.7.43.126:44353
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11032
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.66
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:16.590265 11032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:16.590446 11032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:16.595009 11041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:16.595008 11039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:16.595127 11032 server_base.cc:1034] running on GCE node
W20260806 06:31:16.595192 11038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:16.595386 11032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:16.595587 11032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:16.596725 11032 hybrid_clock.cc:648] HybridClock initialized: now 1785997876596712 us; error 28 us; skew 500 ppm
Aug 06 06:31:16 dist-test-slave-6zbq krb5kdc[10701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997876, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:16.598949 11032 init.cc:377] Logged in from keytab as kudu/127.7.43.66@KRBTEST.COM (short username kudu)
I20260806 06:31:16.599354 11032 webserver.cc:460] Webserver started at http://127.7.43.66:34637/ using document root <none> and password file <none>
I20260806 06:31:16.599534 11032 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:16.599574 11032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:16.599694 11032 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:16.600641 11032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-1/data/instance:
uuid: "5c6e71563c6946e8b8d08f1a0bbf8561"
format_stamp: "Formatted at 2026-08-06 06:31:16 on dist-test-slave-6zbq"
server_key: "a5c50e58069cad08a855a47ec756a9c8"
server_key_iv: "508ef86d8dd9263246a43c7d80ac4799"
server_key_version: "encryptionkey@0"
I20260806 06:31:16.600946 11032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-1/wal/instance:
uuid: "5c6e71563c6946e8b8d08f1a0bbf8561"
format_stamp: "Formatted at 2026-08-06 06:31:16 on dist-test-slave-6zbq"
server_key: "a5c50e58069cad08a855a47ec756a9c8"
server_key_iv: "508ef86d8dd9263246a43c7d80ac4799"
server_key_version: "encryptionkey@0"
I20260806 06:31:16.602108 11032 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 06:31:16.602766 11051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:16.602921 11032 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:16.603003 11032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-1/wal
uuid: "5c6e71563c6946e8b8d08f1a0bbf8561"
format_stamp: "Formatted at 2026-08-06 06:31:16 on dist-test-slave-6zbq"
server_key: "a5c50e58069cad08a855a47ec756a9c8"
server_key_iv: "508ef86d8dd9263246a43c7d80ac4799"
server_key_version: "encryptionkey@0"
I20260806 06:31:16.603052 11032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:16.618242 11032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:16.619793 11032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:16.619890 11032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:16.620051 11032 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:31:16.620266 11032 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:31:16.620293 11032 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:16.620311 11032 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:31:16.620330 11032 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:16.625581 11032 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.66:38029
I20260806 06:31:16.625638 11213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.66:38029 every 8 connection(s)
I20260806 06:31:16.625874 11032 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
Aug 06 06:31:16 dist-test-slave-6zbq krb5kdc[10701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997876, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.66@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:31:16.633276 10817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:16.626504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:38031 (local address 127.7.43.126:44353)
0806 06:31:16.626561 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:16.626563 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:16.626742 (+ 179us) server_negotiation.cc:407] Connection header received
0806 06:31:16.627148 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:16.627149 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:16.627157 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:16.627208 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:16.627993 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:16.628459 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:16.629355 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:16.629528 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:16.629570 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:16.631317 (+ 1747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:16.631319 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:16.631319 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:16.631327 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:16.632324 (+ 997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:16.632641 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:16.632642 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:16.632642 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:16.632685 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:16.632838 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:16.632838 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:16.632838 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:16.632934 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:16.632978 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:16.633159 (+ 181us) server_negotiation.cc:299] Negotiation successful
0806 06:31:16.633217 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 06:31:16.633435 11218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:16.626491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:44353 (local address 127.7.43.66:38031)
0806 06:31:16.626631 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:16.626668 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:16.627076 (+ 408us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:16.627263 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:16.627266 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:16.627763 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:16.627930 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:16.627935 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:16.628516 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:16.628516 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:16.629296 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:16.629298 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:16.629577 (+ 279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:16.629577 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:16.629661 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:16.629999 (+ 338us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:16.630006 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:16.631238 (+ 1232us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:16.632387 (+ 1149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:16.632392 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:16.632394 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:16.632586 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:16.632722 (+ 136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:16.632723 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:16.632723 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:16.632799 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:16.632975 (+ 176us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:16.632978 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:16.633071 (+ 93us) client_negotiation.cc:769] Sending connection context
0806 06:31:16.633143 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 06:31:16.633253 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":76,"threads_started":1}
I20260806 06:31:16.633899 11215 heartbeater.cc:347] Connected to a master server at 127.7.43.126:44353
I20260806 06:31:16.633970 11215 heartbeater.cc:464] Registering TS with master...
I20260806 06:31:16.634148 11215 heartbeater.cc:511] Master 127.7.43.126:44353 requested a full tablet report, sending...
I20260806 06:31:16.634315 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 11032
I20260806 06:31:16.634418 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-1/wal/instance
I20260806 06:31:16.634543 10759 ts_manager.cc:194] Registered new tserver with Master: 5c6e71563c6946e8b8d08f1a0bbf8561 (127.7.43.66:38029)
I20260806 06:31:16.634594 7341 external_mini_cluster.cc:1428] Setting key 8fef24722cb68722827f8e54ed7c83e2
I20260806 06:31:16.635151 10759 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.66@KRBTEST.COM'} at 127.7.43.66:38031
WARNING: no policy specified for kudu/127.7.43.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab.
Entry for principal kudu/127.7.43.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab.
I20260806 06:31:16.656505 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.67:0
--local_ip_for_outbound_sockets=127.7.43.67
--webserver_interface=127.7.43.67
--webserver_port=0
--tserver_master_addrs=127.7.43.126:44353
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab
--principal=kudu/127.7.43.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:34365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:31:16.734144 11224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:16.734385 11224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:16.734417 11224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:16.735538 11224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:16.735574 11224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:16.735608 11224 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.67
I20260806 06:31:16.737025 11224 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:34365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab
--principal=kudu/127.7.43.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.43.67
--webserver_port=0
--tserver_master_addrs=127.7.43.126:44353
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11224
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.67
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:16.737239 11224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:16.737442 11224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:16.741968 11231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:16.741958 11234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:16.742156 11224 server_base.cc:1034] running on GCE node
W20260806 06:31:16.742167 11232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:16.742462 11224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:16.742681 11224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:16.743817 11224 hybrid_clock.cc:648] HybridClock initialized: now 1785997876743804 us; error 24 us; skew 500 ppm
Aug 06 06:31:16 dist-test-slave-6zbq krb5kdc[10701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997876, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:16.746044 11224 init.cc:377] Logged in from keytab as kudu/127.7.43.67@KRBTEST.COM (short username kudu)
I20260806 06:31:16.746455 11224 webserver.cc:460] Webserver started at http://127.7.43.67:38543/ using document root <none> and password file <none>
I20260806 06:31:16.746640 11224 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:16.746685 11224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:16.746801 11224 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:16.747705 11224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-2/data/instance:
uuid: "2302470c4c6c47849fb43bb4410eff35"
format_stamp: "Formatted at 2026-08-06 06:31:16 on dist-test-slave-6zbq"
server_key: "d5f6e2974b8fc372b13ca8e8d9e8151c"
server_key_iv: "d2fa66cd2bd040e9241b09e03708c5a2"
server_key_version: "encryptionkey@0"
I20260806 06:31:16.748009 11224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-2/wal/instance:
uuid: "2302470c4c6c47849fb43bb4410eff35"
format_stamp: "Formatted at 2026-08-06 06:31:16 on dist-test-slave-6zbq"
server_key: "d5f6e2974b8fc372b13ca8e8d9e8151c"
server_key_iv: "d2fa66cd2bd040e9241b09e03708c5a2"
server_key_version: "encryptionkey@0"
I20260806 06:31:16.749157 11224 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:31:16.749778 11245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:16.749920 11224 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:31:16.749998 11224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-2/wal
uuid: "2302470c4c6c47849fb43bb4410eff35"
format_stamp: "Formatted at 2026-08-06 06:31:16 on dist-test-slave-6zbq"
server_key: "d5f6e2974b8fc372b13ca8e8d9e8151c"
server_key_iv: "d2fa66cd2bd040e9241b09e03708c5a2"
server_key_version: "encryptionkey@0"
I20260806 06:31:16.750058 11224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:16.763772 11224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:16.765264 11224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:16.765383 11224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:16.765554 11224 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:31:16.765816 11224 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:31:16.765848 11224 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:16.765872 11224 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:31:16.765890 11224 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:16.771324 11224 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.67:40407
I20260806 06:31:16.771394 11417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.67:40407 every 8 connection(s)
I20260806 06:31:16.771644 11224 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
Aug 06 06:31:16 dist-test-slave-6zbq krb5kdc[10701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997876, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.67@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:31:16.779623 10817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:16.772374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:57893 (local address 127.7.43.126:44353)
0806 06:31:16.772429 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:16.772431 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:16.772597 (+ 166us) server_negotiation.cc:407] Connection header received
0806 06:31:16.773009 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:16.773010 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:16.773018 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:16.773069 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:16.773890 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:16.774419 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:16.775305 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:16.775483 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:16.775523 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:16.777369 (+ 1846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:16.777371 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:16.777371 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:16.777377 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:16.778567 (+ 1190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:16.778875 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:16.778876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:16.778876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:16.778920 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:16.779081 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:16.779081 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:16.779082 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:16.779176 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:16.779220 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:16.779507 (+ 287us) server_negotiation.cc:299] Negotiation successful
0806 06:31:16.779562 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 06:31:16.779762 11420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:16.772328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:44353 (local address 127.7.43.67:57893)
0806 06:31:16.772487 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:16.772525 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:16.772938 (+ 413us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:16.773117 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:16.773120 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:16.773646 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:16.773823 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:16.773828 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:16.774480 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:16.774480 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:16.775245 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:16.775247 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:16.775531 (+ 284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:16.775531 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:16.775621 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:16.775983 (+ 362us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:16.775990 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:16.777287 (+ 1297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:16.778629 (+ 1342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:16.778633 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:16.778634 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:16.778825 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:16.778961 (+ 136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:16.778962 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:16.778962 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:16.779042 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:16.779282 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:16.779289 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:16.779398 (+ 109us) client_negotiation.cc:769] Sending connection context
0806 06:31:16.779484 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 06:31:16.779595 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":68,"threads_started":1}
I20260806 06:31:16.780206 11418 heartbeater.cc:347] Connected to a master server at 127.7.43.126:44353
I20260806 06:31:16.780292 11418 heartbeater.cc:464] Registering TS with master...
I20260806 06:31:16.780445 11418 heartbeater.cc:511] Master 127.7.43.126:44353 requested a full tablet report, sending...
I20260806 06:31:16.780532 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 11224
I20260806 06:31:16.780627 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-2/wal/instance
I20260806 06:31:16.780824 7341 external_mini_cluster.cc:1428] Setting key ffdcc8bd61a5e9589b1682c2f3c23f36
I20260806 06:31:16.780830 10758 ts_manager.cc:194] Registered new tserver with Master: 2302470c4c6c47849fb43bb4410eff35 (127.7.43.67:40407)
I20260806 06:31:16.781263 10758 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.67@KRBTEST.COM'} at 127.7.43.67:57893
I20260806 06:31:16.781641 7341 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 06:31:16.793910 10817 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:16.793040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54266 (local address 127.7.43.126:44353)
0806 06:31:16.793092 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:16.793094 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:16.793254 (+ 160us) server_negotiation.cc:407] Connection header received
0806 06:31:16.793276 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:16.793277 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:16.793286 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:16.793324 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:16.793850 (+ 526us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54266: BlockingRecv error: recv got EOF from 127.0.0.1:54266 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 06:31:16.794863 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 10845
I20260806 06:31:16.809939 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 11032
I20260806 06:31:16.821147 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 11224
I20260806 06:31:16.827114 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 10716
2026-08-06T06:31:16Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1843 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785997861828271-7341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:31:16 dist-test-slave-6zbq krb5kdc[11434](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:31:16 dist-test-slave-6zbq krb5kdc[11434](info): set up 2 sockets
Aug 06 06:31:16 dist-test-slave-6zbq krb5kdc[11434](info): commencing operation
krb5kdc: starting...
W20260806 06:31:17.944844 7341 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Aug 06 06:31:18 dist-test-slave-6zbq krb5kdc[11434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997878, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:31:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:31:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.43.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
I20260806 06:31:18.054738 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:37043
--webserver_interface=127.7.43.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:44243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.43.126:37043
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:31:18.133049 11448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:18.133234 11448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:18.133271 11448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:18.134392 11448 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:31:18.134428 11448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:18.134447 11448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:18.134464 11448 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:31:18.134487 11448 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:31:18.135941 11448 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:44243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785997861828271-7341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.43.126:37043
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:37043
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.43.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11448
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:18.136162 11448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:18.136354 11448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:31:18.140872 11448 server_base.cc:1034] running on GCE node
W20260806 06:31:18.140841 11454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:18.140796 11461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:18.140803 11455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:18.141259 11448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:18.141505 11448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:18.142630 11448 hybrid_clock.cc:648] HybridClock initialized: now 1785997878142614 us; error 32 us; skew 500 ppm
Aug 06 06:31:18 dist-test-slave-6zbq krb5kdc[11434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997878, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:18.144784 11448 init.cc:377] Logged in from keytab as kudu/127.7.43.126@KRBTEST.COM (short username kudu)
I20260806 06:31:18.145172 11448 webserver.cc:460] Webserver started at http://127.7.43.126:35187/ using document root <none> and password file <none>
I20260806 06:31:18.145375 11448 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:18.145421 11448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:18.145541 11448 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:18.146520 11448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785997861828271-7341-0/minicluster-data/master-0/data/instance:
uuid: "825866dc76f44c9cbddcecdf8438fc6b"
format_stamp: "Formatted at 2026-08-06 06:31:18 on dist-test-slave-6zbq"
server_key: "7ff213058c69ce0e448ee0b30166c5d2"
server_key_iv: "96d077d68a71ca6045c36ceb8427a3b9"
server_key_version: "encryptionkey@0"
I20260806 06:31:18.146831 11448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785997861828271-7341-0/minicluster-data/master-0/wal/instance:
uuid: "825866dc76f44c9cbddcecdf8438fc6b"
format_stamp: "Formatted at 2026-08-06 06:31:18 on dist-test-slave-6zbq"
server_key: "7ff213058c69ce0e448ee0b30166c5d2"
server_key_iv: "96d077d68a71ca6045c36ceb8427a3b9"
server_key_version: "encryptionkey@0"
I20260806 06:31:18.147955 11448 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:31:18.148604 11470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:18.148747 11448 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:31:18.148826 11448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "825866dc76f44c9cbddcecdf8438fc6b"
format_stamp: "Formatted at 2026-08-06 06:31:18 on dist-test-slave-6zbq"
server_key: "7ff213058c69ce0e448ee0b30166c5d2"
server_key_iv: "96d077d68a71ca6045c36ceb8427a3b9"
server_key_version: "encryptionkey@0"
I20260806 06:31:18.148883 11448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:18.177351 11448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:18.179008 11448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:18.179131 11448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:18.182770 11448 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.126:37043
I20260806 06:31:18.182806 11542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.126:37043 every 8 connection(s)
I20260806 06:31:18.183115 11448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:31:18.183889 11543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:18.185993 11543 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 825866dc76f44c9cbddcecdf8438fc6b: Bootstrap starting.
I20260806 06:31:18.186532 11543 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 825866dc76f44c9cbddcecdf8438fc6b: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:18.186767 11543 log.cc:824] T 00000000000000000000000000000000 P 825866dc76f44c9cbddcecdf8438fc6b: Log is configured to *not* fsync() on all Append() calls
I20260806 06:31:18.187333 11543 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 825866dc76f44c9cbddcecdf8438fc6b: No bootstrap required, opened a new log
I20260806 06:31:18.188634 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 11448
I20260806 06:31:18.188710 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785997861828271-7341-0/minicluster-data/master-0/wal/instance
I20260806 06:31:18.188875 7341 external_mini_cluster.cc:1428] Setting key 55d8392fa643e4246ea4ca992b4ceff8
I20260806 06:31:18.188862 11543 raft_consensus.cc:348] T 00000000000000000000000000000000 P 825866dc76f44c9cbddcecdf8438fc6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "825866dc76f44c9cbddcecdf8438fc6b" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 37043 } }
I20260806 06:31:18.189010 11543 raft_consensus.cc:374] T 00000000000000000000000000000000 P 825866dc76f44c9cbddcecdf8438fc6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:18.189051 11543 raft_consensus.cc:729] T 00000000000000000000000000000000 P 825866dc76f44c9cbddcecdf8438fc6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 825866dc76f44c9cbddcecdf8438fc6b, State: Initialized, Role: FOLLOWER
I20260806 06:31:18.189196 11543 consensus_queue.cc:260] T 00000000000000000000000000000000 P 825866dc76f44c9cbddcecdf8438fc6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "825866dc76f44c9cbddcecdf8438fc6b" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 37043 } }
I20260806 06:31:18.189268 11543 raft_consensus.cc:388] T 00000000000000000000000000000000 P 825866dc76f44c9cbddcecdf8438fc6b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:31:18.189298 11543 raft_consensus.cc:482] T 00000000000000000000000000000000 P 825866dc76f44c9cbddcecdf8438fc6b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:31:18.189361 11543 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 825866dc76f44c9cbddcecdf8438fc6b [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:18.189908 11543 raft_consensus.cc:504] T 00000000000000000000000000000000 P 825866dc76f44c9cbddcecdf8438fc6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "825866dc76f44c9cbddcecdf8438fc6b" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 37043 } }
I20260806 06:31:18.190016 11543 leader_election.cc:302] T 00000000000000000000000000000000 P 825866dc76f44c9cbddcecdf8438fc6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 825866dc76f44c9cbddcecdf8438fc6b; no voters:
I20260806 06:31:18.190173 11543 leader_election.cc:288] T 00000000000000000000000000000000 P 825866dc76f44c9cbddcecdf8438fc6b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:31:18.190212 11549 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 825866dc76f44c9cbddcecdf8438fc6b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:31:18.190429 11549 raft_consensus.cc:686] T 00000000000000000000000000000000 P 825866dc76f44c9cbddcecdf8438fc6b [term 1 LEADER]: Becoming Leader. State: Replica: 825866dc76f44c9cbddcecdf8438fc6b, State: Running, Role: LEADER
I20260806 06:31:18.190521 11543 sys_catalog.cc:564] T 00000000000000000000000000000000 P 825866dc76f44c9cbddcecdf8438fc6b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:31:18.190605 11549 consensus_queue.cc:237] T 00000000000000000000000000000000 P 825866dc76f44c9cbddcecdf8438fc6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "825866dc76f44c9cbddcecdf8438fc6b" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 37043 } }
I20260806 06:31:18.190975 11550 sys_catalog.cc:455] T 00000000000000000000000000000000 P 825866dc76f44c9cbddcecdf8438fc6b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "825866dc76f44c9cbddcecdf8438fc6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "825866dc76f44c9cbddcecdf8438fc6b" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 37043 } } }
I20260806 06:31:18.191071 11550 sys_catalog.cc:458] T 00000000000000000000000000000000 P 825866dc76f44c9cbddcecdf8438fc6b [sys.catalog]: This master's current role is: LEADER
I20260806 06:31:18.190981 11552 sys_catalog.cc:455] T 00000000000000000000000000000000 P 825866dc76f44c9cbddcecdf8438fc6b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 825866dc76f44c9cbddcecdf8438fc6b. Latest consensus state: current_term: 1 leader_uuid: "825866dc76f44c9cbddcecdf8438fc6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "825866dc76f44c9cbddcecdf8438fc6b" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 37043 } } }
I20260806 06:31:18.191293 11552 sys_catalog.cc:458] T 00000000000000000000000000000000 P 825866dc76f44c9cbddcecdf8438fc6b [sys.catalog]: This master's current role is: LEADER
I20260806 06:31:18.191246 11560 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:31:18.191792 11560 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:31:18.193342 11560 catalog_manager.cc:1347] Generated new cluster ID: 6dc7e93d013946cd9ebbd158ad416d47
I20260806 06:31:18.193403 11560 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:31:18 dist-test-slave-6zbq krb5kdc[11434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997878, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:31:18.200742 11547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:18.189257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59172 (local address 127.7.43.126:37043)
0806 06:31:18.189779 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:18.189788 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:18.189821 (+ 33us) server_negotiation.cc:407] Connection header received
0806 06:31:18.189938 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:18.189945 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:18.190001 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:18.190148 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:18.190930 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:18.192335 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:18.194547 (+ 2212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:18.195058 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:18.195157 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:18.197220 (+ 2063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:18.197224 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:18.197225 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:18.197236 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:18.199385 (+ 2149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:18.199720 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:18.199723 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:18.199724 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:18.199806 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:18.199977 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:18.199978 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:18.199978 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:18.200204 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:18.200349 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:18.200452 (+ 103us) server_negotiation.cc:299] Negotiation successful
0806 06:31:18.200581 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":407,"threads_started":1}
I20260806 06:31:18.201527 7341 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:31:18.206676 11560 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:31:18.207129 11560 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:31:18.213269 11560 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 825866dc76f44c9cbddcecdf8438fc6b: Generated new TSK 0
I20260806 06:31:18.213428 11560 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:31:18.228467 11547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:18.225070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59182 (local address 127.7.43.126:37043)
0806 06:31:18.225112 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:18.225114 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:18.225216 (+ 102us) server_negotiation.cc:407] Connection header received
0806 06:31:18.225297 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:18.225298 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:18.225306 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:18.225343 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:18.225742 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:18.226223 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:18.227281 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:18.227467 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:18.227514 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:18.228090 (+ 576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:18.228092 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:18.228092 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:31:18.228270 (+ 178us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:18.228285 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:31:18.228291 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:18.228334 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:18.228400 (+ 66us) server_negotiation.cc:299] Negotiation successful
0806 06:31:18.228418 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:31:18.230793 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 11448
2026-08-06T06:31:18Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1407 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:31:18 dist-test-slave-6zbq krb5kdc[11598](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:31:18 dist-test-slave-6zbq krb5kdc[11598](info): set up 2 sockets
Aug 06 06:31:18 dist-test-slave-6zbq krb5kdc[11598](info): commencing operation
krb5kdc: starting...
W20260806 06:31:19.335633 7341 mini_kdc.cc:121] Time spent starting KDC: real 1.078s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Aug 06 06:31:19 dist-test-slave-6zbq krb5kdc[11598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:31:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:31:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.43.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
I20260806 06:31:19.440784 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:46083
--webserver_interface=127.7.43.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:44309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.43.126:46083
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:31:19.517935 11620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:19.518115 11620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:19.518136 11620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:19.519467 11620 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:31:19.519515 11620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:19.519528 11620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:19.519562 11620 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:31:19.519589 11620 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:31:19.521078 11620 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:44309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.43.126:46083
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:46083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.43.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11620
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:19.521304 11620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:19.521507 11620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:19.526008 11630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:19.525995 11629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:19.525995 11632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:19.526362 11620 server_base.cc:1034] running on GCE node
I20260806 06:31:19.526580 11620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:19.526841 11620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:19.527987 11620 hybrid_clock.cc:648] HybridClock initialized: now 1785997879527975 us; error 28 us; skew 500 ppm
Aug 06 06:31:19 dist-test-slave-6zbq krb5kdc[11598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997879, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:19.530211 11620 init.cc:377] Logged in from keytab as kudu/127.7.43.126@KRBTEST.COM (short username kudu)
I20260806 06:31:19.530640 11620 webserver.cc:460] Webserver started at http://127.7.43.126:43229/ using document root <none> and password file <none>
I20260806 06:31:19.530851 11620 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:19.530898 11620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:19.531016 11620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:19.531961 11620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/master-0/data/instance:
uuid: "5d85f139497d4408bffc4e4604c71d84"
format_stamp: "Formatted at 2026-08-06 06:31:19 on dist-test-slave-6zbq"
server_key: "7e219be66827042c2dfe91748f4765ff"
server_key_iv: "069092d0e351caf1d92eb9d1d7429e12"
server_key_version: "encryptionkey@0"
I20260806 06:31:19.532289 11620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/master-0/wal/instance:
uuid: "5d85f139497d4408bffc4e4604c71d84"
format_stamp: "Formatted at 2026-08-06 06:31:19 on dist-test-slave-6zbq"
server_key: "7e219be66827042c2dfe91748f4765ff"
server_key_iv: "069092d0e351caf1d92eb9d1d7429e12"
server_key_version: "encryptionkey@0"
I20260806 06:31:19.533452 11620 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:31:19.534034 11641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:19.534216 11620 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:31:19.534298 11620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "5d85f139497d4408bffc4e4604c71d84"
format_stamp: "Formatted at 2026-08-06 06:31:19 on dist-test-slave-6zbq"
server_key: "7e219be66827042c2dfe91748f4765ff"
server_key_iv: "069092d0e351caf1d92eb9d1d7429e12"
server_key_version: "encryptionkey@0"
I20260806 06:31:19.534389 11620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:19.559875 11620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:19.561400 11620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:19.561520 11620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:19.565155 11620 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.126:46083
I20260806 06:31:19.565184 11716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.126:46083 every 8 connection(s)
I20260806 06:31:19.565474 11620 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:31:19.566599 11718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:19.568647 11718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d85f139497d4408bffc4e4604c71d84: Bootstrap starting.
I20260806 06:31:19.569180 11718 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d85f139497d4408bffc4e4604c71d84: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:19.569425 11718 log.cc:824] T 00000000000000000000000000000000 P 5d85f139497d4408bffc4e4604c71d84: Log is configured to *not* fsync() on all Append() calls
I20260806 06:31:19.570025 11718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d85f139497d4408bffc4e4604c71d84: No bootstrap required, opened a new log
I20260806 06:31:19.571539 11718 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d85f139497d4408bffc4e4604c71d84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d85f139497d4408bffc4e4604c71d84" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 46083 } }
I20260806 06:31:19.571669 11718 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d85f139497d4408bffc4e4604c71d84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:19.571689 11718 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d85f139497d4408bffc4e4604c71d84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d85f139497d4408bffc4e4604c71d84, State: Initialized, Role: FOLLOWER
I20260806 06:31:19.571830 11718 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d85f139497d4408bffc4e4604c71d84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d85f139497d4408bffc4e4604c71d84" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 46083 } }
I20260806 06:31:19.571908 11718 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d85f139497d4408bffc4e4604c71d84 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:31:19.571943 11718 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d85f139497d4408bffc4e4604c71d84 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:31:19.571987 11718 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d85f139497d4408bffc4e4604c71d84 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:19.572726 11718 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d85f139497d4408bffc4e4604c71d84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d85f139497d4408bffc4e4604c71d84" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 46083 } }
I20260806 06:31:19.572815 11718 leader_election.cc:302] T 00000000000000000000000000000000 P 5d85f139497d4408bffc4e4604c71d84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d85f139497d4408bffc4e4604c71d84; no voters:
I20260806 06:31:19.572956 11718 leader_election.cc:288] T 00000000000000000000000000000000 P 5d85f139497d4408bffc4e4604c71d84 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:31:19.572994 11723 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d85f139497d4408bffc4e4604c71d84 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:31:19.573114 11723 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d85f139497d4408bffc4e4604c71d84 [term 1 LEADER]: Becoming Leader. State: Replica: 5d85f139497d4408bffc4e4604c71d84, State: Running, Role: LEADER
I20260806 06:31:19.573186 11718 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d85f139497d4408bffc4e4604c71d84 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:31:19.573222 11723 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d85f139497d4408bffc4e4604c71d84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d85f139497d4408bffc4e4604c71d84" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 46083 } }
I20260806 06:31:19.573557 11725 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d85f139497d4408bffc4e4604c71d84 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5d85f139497d4408bffc4e4604c71d84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d85f139497d4408bffc4e4604c71d84" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 46083 } } }
I20260806 06:31:19.573575 11726 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d85f139497d4408bffc4e4604c71d84 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d85f139497d4408bffc4e4604c71d84. Latest consensus state: current_term: 1 leader_uuid: "5d85f139497d4408bffc4e4604c71d84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d85f139497d4408bffc4e4604c71d84" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 46083 } } }
I20260806 06:31:19.573686 11725 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d85f139497d4408bffc4e4604c71d84 [sys.catalog]: This master's current role is: LEADER
I20260806 06:31:19.573710 11726 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d85f139497d4408bffc4e4604c71d84 [sys.catalog]: This master's current role is: LEADER
W20260806 06:31:19.574739 11752 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5d85f139497d4408bffc4e4604c71d84: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:31:19.574782 11752 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:31:19.574757 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 11620
I20260806 06:31:19.574838 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/master-0/wal/instance
I20260806 06:31:19.574832 11739 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:31:19.574973 11739 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:31:19.575009 7341 external_mini_cluster.cc:1428] Setting key 540bb1cc420d2e0607d4bb5ea56d4fd5
I20260806 06:31:19.576288 11739 catalog_manager.cc:1347] Generated new cluster ID: 13cb67dd6af84340924eab47e919da40
I20260806 06:31:19.576328 11739 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:31:19.579635 11739 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:31:19.580282 11739 catalog_manager.cc:1504] Loading token signing keys...
Aug 06 06:31:19 dist-test-slave-6zbq krb5kdc[11598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:31:19.589047 11739 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d85f139497d4408bffc4e4604c71d84: Generated new TSK 0
I20260806 06:31:19.589109 11758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:19.575403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52730 (local address 127.7.43.126:46083)
0806 06:31:19.575546 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:19.575554 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:19.578559 (+ 3005us) server_negotiation.cc:407] Connection header received
0806 06:31:19.578681 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:19.578687 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:19.578737 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:19.578863 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:19.579582 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:19.581583 (+ 2001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:19.582609 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:19.583166 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:19.583254 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:19.585823 (+ 2569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:19.585827 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:19.585827 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:19.585839 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:19.587638 (+ 1799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:19.587993 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:19.587996 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:19.587996 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:19.588098 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:19.588265 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:19.588266 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:19.588266 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:19.588516 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:19.588652 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:19.588753 (+ 101us) server_negotiation.cc:299] Negotiation successful
0806 06:31:19.588893 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":65,"threads_started":1}
I20260806 06:31:19.589213 11739 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.43.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab.
Entry for principal kudu/127.7.43.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab.
I20260806 06:31:19.611120 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.65:0
--local_ip_for_outbound_sockets=127.7.43.65
--webserver_interface=127.7.43.65
--webserver_port=0
--tserver_master_addrs=127.7.43.126:46083
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab
--principal=kudu/127.7.43.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:44309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:31:19.688656 11764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:19.688829 11764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:19.688860 11764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:19.689945 11764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:19.689983 11764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:19.690016 11764 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.65
I20260806 06:31:19.691407 11764 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:44309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab
--principal=kudu/127.7.43.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.43.65
--webserver_port=0
--tserver_master_addrs=127.7.43.126:46083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11764
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.65
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:19.691622 11764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:19.691818 11764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:19.696311 11769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:19.696323 11770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:19.696321 11772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:19.696485 11764 server_base.cc:1034] running on GCE node
I20260806 06:31:19.696682 11764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:19.696885 11764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:19.698025 11764 hybrid_clock.cc:648] HybridClock initialized: now 1785997879698014 us; error 28 us; skew 500 ppm
Aug 06 06:31:19 dist-test-slave-6zbq krb5kdc[11598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997879, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:19.700206 11764 init.cc:377] Logged in from keytab as kudu/127.7.43.65@KRBTEST.COM (short username kudu)
I20260806 06:31:19.700596 11764 webserver.cc:460] Webserver started at http://127.7.43.65:36695/ using document root <none> and password file <none>
I20260806 06:31:19.700803 11764 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:19.700847 11764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:19.700965 11764 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:19.701894 11764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-0/data/instance:
uuid: "4ddab331234c4e6fb16b203802f5101a"
format_stamp: "Formatted at 2026-08-06 06:31:19 on dist-test-slave-6zbq"
server_key: "21d84b88d79378d157384dc70ef8c2ea"
server_key_iv: "08c91a56ea4c91fc2a883ce1f4a3c788"
server_key_version: "encryptionkey@0"
I20260806 06:31:19.702194 11764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-0/wal/instance:
uuid: "4ddab331234c4e6fb16b203802f5101a"
format_stamp: "Formatted at 2026-08-06 06:31:19 on dist-test-slave-6zbq"
server_key: "21d84b88d79378d157384dc70ef8c2ea"
server_key_iv: "08c91a56ea4c91fc2a883ce1f4a3c788"
server_key_version: "encryptionkey@0"
I20260806 06:31:19.703375 11764 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:31:19.704008 11784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:19.704152 11764 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:31:19.704231 11764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-0/wal
uuid: "4ddab331234c4e6fb16b203802f5101a"
format_stamp: "Formatted at 2026-08-06 06:31:19 on dist-test-slave-6zbq"
server_key: "21d84b88d79378d157384dc70ef8c2ea"
server_key_iv: "08c91a56ea4c91fc2a883ce1f4a3c788"
server_key_version: "encryptionkey@0"
I20260806 06:31:19.704293 11764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:19.734481 11764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:19.736022 11764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:19.736145 11764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:19.736328 11764 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:31:19.736581 11764 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:31:19.736613 11764 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:19.736646 11764 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:31:19.736665 11764 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:19.742059 11764 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.65:41037
I20260806 06:31:19.742117 11946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.65:41037 every 8 connection(s)
I20260806 06:31:19.742393 11764 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
I20260806 06:31:19.745492 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 11764
I20260806 06:31:19.745566 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-0/wal/instance
I20260806 06:31:19.745714 7341 external_mini_cluster.cc:1428] Setting key 0bf261a2fdb952fb7d1267ed24d2e8c0
Aug 06 06:31:19 dist-test-slave-6zbq krb5kdc[11598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997879, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.65@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:31:19.751119 11758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:19.742989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:55807 (local address 127.7.43.126:46083)
0806 06:31:19.743036 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:19.743038 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:19.743215 (+ 177us) server_negotiation.cc:407] Connection header received
0806 06:31:19.743616 (+ 401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:19.743617 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:19.743627 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:19.743676 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:19.744449 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:19.744935 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:19.745896 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:19.746152 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:19.746203 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:19.748336 (+ 2133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:19.748338 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:19.748338 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:19.748346 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:19.749665 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:19.750094 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:19.750096 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:19.750096 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:19.750168 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:19.750456 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:19.750457 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:19.750458 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:19.750594 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:19.750664 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:19.750944 (+ 280us) server_negotiation.cc:299] Negotiation successful
0806 06:31:19.751030 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 06:31:19.751155 11950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:19.742951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:46083 (local address 127.7.43.65:55807)
0806 06:31:19.743095 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:19.743132 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:19.743545 (+ 413us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:19.743723 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:19.743725 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:19.744216 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:19.744386 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:19.744391 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:19.744993 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:19.744993 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:19.745804 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:19.745807 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:19.746232 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:19.746233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:19.746395 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:19.746819 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:19.746827 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:19.748252 (+ 1425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:19.749752 (+ 1500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:19.749760 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:19.749762 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:19.750020 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:19.750238 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:19.750239 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:19.750240 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:19.750385 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:19.750662 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:19.750669 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:19.750821 (+ 152us) client_negotiation.cc:769] Sending connection context
0806 06:31:19.750903 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 06:31:19.751014 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":78,"threads_started":1}
I20260806 06:31:19.751592 11948 heartbeater.cc:347] Connected to a master server at 127.7.43.126:46083
I20260806 06:31:19.751677 11948 heartbeater.cc:464] Registering TS with master...
I20260806 06:31:19.751842 11948 heartbeater.cc:511] Master 127.7.43.126:46083 requested a full tablet report, sending...
I20260806 06:31:19.752395 11666 ts_manager.cc:194] Registered new tserver with Master: 4ddab331234c4e6fb16b203802f5101a (127.7.43.65:41037)
I20260806 06:31:19.753242 11666 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.65@KRBTEST.COM'} at 127.7.43.65:55807
WARNING: no policy specified for kudu/127.7.43.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab.
Entry for principal kudu/127.7.43.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab.
I20260806 06:31:19.769378 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.66:0
--local_ip_for_outbound_sockets=127.7.43.66
--webserver_interface=127.7.43.66
--webserver_port=0
--tserver_master_addrs=127.7.43.126:46083
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab
--principal=kudu/127.7.43.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:44309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:31:19.847177 11954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:19.847355 11954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:19.847376 11954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:19.848444 11954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:19.848476 11954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:19.848500 11954 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.66
I20260806 06:31:19.849823 11954 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:44309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab
--principal=kudu/127.7.43.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.43.66
--webserver_port=0
--tserver_master_addrs=127.7.43.126:46083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11954
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.66
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:19.849996 11954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:19.850155 11954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:31:19.854655 11954 server_base.cc:1034] running on GCE node
W20260806 06:31:19.854597 11962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:19.854619 11965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:19.854601 11963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:19.854979 11954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:19.855191 11954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:19.856340 11954 hybrid_clock.cc:648] HybridClock initialized: now 1785997879856324 us; error 29 us; skew 500 ppm
Aug 06 06:31:19 dist-test-slave-6zbq krb5kdc[11598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997879, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:19.858677 11954 init.cc:377] Logged in from keytab as kudu/127.7.43.66@KRBTEST.COM (short username kudu)
I20260806 06:31:19.859107 11954 webserver.cc:460] Webserver started at http://127.7.43.66:37001/ using document root <none> and password file <none>
I20260806 06:31:19.859305 11954 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:19.859354 11954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:19.859476 11954 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:19.860558 11954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-1/data/instance:
uuid: "1d001748cad24dd4b95b289ec45104bf"
format_stamp: "Formatted at 2026-08-06 06:31:19 on dist-test-slave-6zbq"
server_key: "40117b42b139d26d6f19e534e0d21a2b"
server_key_iv: "4a38dd2052c97807332aa0da73d83150"
server_key_version: "encryptionkey@0"
I20260806 06:31:19.860896 11954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-1/wal/instance:
uuid: "1d001748cad24dd4b95b289ec45104bf"
format_stamp: "Formatted at 2026-08-06 06:31:19 on dist-test-slave-6zbq"
server_key: "40117b42b139d26d6f19e534e0d21a2b"
server_key_iv: "4a38dd2052c97807332aa0da73d83150"
server_key_version: "encryptionkey@0"
I20260806 06:31:19.862095 11954 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 06:31:19.862855 11975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:19.863016 11954 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:19.863099 11954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-1/wal
uuid: "1d001748cad24dd4b95b289ec45104bf"
format_stamp: "Formatted at 2026-08-06 06:31:19 on dist-test-slave-6zbq"
server_key: "40117b42b139d26d6f19e534e0d21a2b"
server_key_iv: "4a38dd2052c97807332aa0da73d83150"
server_key_version: "encryptionkey@0"
I20260806 06:31:19.863165 11954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:19.884444 11954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:19.886052 11954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:19.886178 11954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:19.886400 11954 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:31:19.886677 11954 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:31:19.886709 11954 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:19.886735 11954 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:31:19.886772 11954 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:19.892257 11954 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.66:34047
I20260806 06:31:19.892295 12133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.66:34047 every 8 connection(s)
I20260806 06:31:19.892567 11954 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
I20260806 06:31:19.893508 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 11954
I20260806 06:31:19.893589 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-1/wal/instance
I20260806 06:31:19.893762 7341 external_mini_cluster.cc:1428] Setting key 6a3b51689b13f8474533cf1ecaf83001
Aug 06 06:31:19 dist-test-slave-6zbq krb5kdc[11598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997879, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.66@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:31:19.901571 11758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:19.893249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:53513 (local address 127.7.43.126:46083)
0806 06:31:19.893315 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:19.893317 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:19.893484 (+ 167us) server_negotiation.cc:407] Connection header received
0806 06:31:19.894044 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:19.894046 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:19.894058 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:19.894105 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:19.895027 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:19.895502 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:19.896816 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:19.897011 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:19.897069 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:19.899151 (+ 2082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:19.899152 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:19.899153 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:19.899160 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:19.900338 (+ 1178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:19.900684 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:19.900685 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:19.900685 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:19.900740 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:19.900971 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:19.900972 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:19.900972 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:19.901079 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:19.901147 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:19.901401 (+ 254us) server_negotiation.cc:299] Negotiation successful
0806 06:31:19.901482 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 06:31:19.901748 12140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:19.893214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:46083 (local address 127.7.43.66:53513)
0806 06:31:19.893366 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:19.893405 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:19.893952 (+ 547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:19.894180 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:19.894184 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:19.894763 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:19.894939 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:19.894945 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:19.895582 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:19.895583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:19.896729 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:19.896732 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:19.897087 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:19.897087 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:19.897192 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:19.897731 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:19.897740 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:19.899064 (+ 1324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:19.900409 (+ 1345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:19.900416 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:19.900418 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:19.900619 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:19.900796 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:19.900797 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:19.900797 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:19.900915 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:19.901146 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:19.901153 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:19.901286 (+ 133us) client_negotiation.cc:769] Sending connection context
0806 06:31:19.901370 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 06:31:19.901512 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":83,"threads_started":1}
WARNING: no policy specified for kudu/127.7.43.67@KRBTEST.COM; defaulting to no policy
I20260806 06:31:19.902170 12134 heartbeater.cc:347] Connected to a master server at 127.7.43.126:46083
I20260806 06:31:19.902252 12134 heartbeater.cc:464] Registering TS with master...
I20260806 06:31:19.902444 12134 heartbeater.cc:511] Master 127.7.43.126:46083 requested a full tablet report, sending...
I20260806 06:31:19.902783 11666 ts_manager.cc:194] Registered new tserver with Master: 1d001748cad24dd4b95b289ec45104bf (127.7.43.66:34047)
I20260806 06:31:19.903160 11666 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.66@KRBTEST.COM'} at 127.7.43.66:53513
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab.
Entry for principal kudu/127.7.43.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab.
I20260806 06:31:19.916953 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.67:0
--local_ip_for_outbound_sockets=127.7.43.67
--webserver_interface=127.7.43.67
--webserver_port=0
--tserver_master_addrs=127.7.43.126:46083
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab
--principal=kudu/127.7.43.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:44309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:31:19.996289 12143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:19.996469 12143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:19.996490 12143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:19.997604 12143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:19.997646 12143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:19.997732 12143 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.67
I20260806 06:31:19.999325 12143 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:44309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab
--principal=kudu/127.7.43.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.43.67
--webserver_port=0
--tserver_master_addrs=127.7.43.126:46083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12143
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.67
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:19.999522 12143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:19.999717 12143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:20.004287 12152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:20.004293 12148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:20.004293 12149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:20.004452 12143 server_base.cc:1034] running on GCE node
I20260806 06:31:20.004627 12143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:20.004813 12143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:20.005942 12143 hybrid_clock.cc:648] HybridClock initialized: now 1785997880005929 us; error 28 us; skew 500 ppm
Aug 06 06:31:20 dist-test-slave-6zbq krb5kdc[11598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997880, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:20.008311 12143 init.cc:377] Logged in from keytab as kudu/127.7.43.67@KRBTEST.COM (short username kudu)
I20260806 06:31:20.008719 12143 webserver.cc:460] Webserver started at http://127.7.43.67:35425/ using document root <none> and password file <none>
I20260806 06:31:20.008904 12143 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:20.008945 12143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:20.009064 12143 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:20.010010 12143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-2/data/instance:
uuid: "0fa8bf6518854f7bb78de37f49ac61cb"
format_stamp: "Formatted at 2026-08-06 06:31:20 on dist-test-slave-6zbq"
server_key: "adb45fb8d4b80b70b6d7a6aa1431e786"
server_key_iv: "c649bbc2645a3d56a2f0e356119935e5"
server_key_version: "encryptionkey@0"
I20260806 06:31:20.010318 12143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-2/wal/instance:
uuid: "0fa8bf6518854f7bb78de37f49ac61cb"
format_stamp: "Formatted at 2026-08-06 06:31:20 on dist-test-slave-6zbq"
server_key: "adb45fb8d4b80b70b6d7a6aa1431e786"
server_key_iv: "c649bbc2645a3d56a2f0e356119935e5"
server_key_version: "encryptionkey@0"
I20260806 06:31:20.011546 12143 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:31:20.012163 12164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:20.012319 12143 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:31:20.012399 12143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-2/wal
uuid: "0fa8bf6518854f7bb78de37f49ac61cb"
format_stamp: "Formatted at 2026-08-06 06:31:20 on dist-test-slave-6zbq"
server_key: "adb45fb8d4b80b70b6d7a6aa1431e786"
server_key_iv: "c649bbc2645a3d56a2f0e356119935e5"
server_key_version: "encryptionkey@0"
I20260806 06:31:20.012460 12143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:20.023867 12143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:20.025760 12143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:20.025887 12143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:20.026075 12143 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:31:20.026372 12143 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:31:20.026407 12143 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:20.026433 12143 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:31:20.026450 12143 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:20.031978 12143 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.67:33755
I20260806 06:31:20.032013 12328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.67:33755 every 8 connection(s)
I20260806 06:31:20.032279 12143 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
I20260806 06:31:20.041191 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 12143
I20260806 06:31:20.041265 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785997861828271-7341-0/minicluster-data/ts-2/wal/instance
I20260806 06:31:20.041410 7341 external_mini_cluster.cc:1428] Setting key 879e7592fe92215a9cfd8c803e1bcdac
Aug 06 06:31:20 dist-test-slave-6zbq krb5kdc[11598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997880, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.67@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:31:20.044521 11758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:20.033127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:37147 (local address 127.7.43.126:46083)
0806 06:31:20.033173 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:20.033175 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:20.033240 (+ 65us) server_negotiation.cc:407] Connection header received
0806 06:31:20.033715 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:20.033716 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:20.033725 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:20.033770 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:20.034620 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:20.035103 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:20.036031 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:20.036218 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:20.036282 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:20.042457 (+ 6175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:20.042458 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:20.042459 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:20.042467 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:20.043524 (+ 1057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:20.043876 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:20.043876 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:20.043877 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:20.043919 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:20.044086 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:20.044087 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:20.044087 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:20.044174 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:20.044216 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:20.044402 (+ 186us) server_negotiation.cc:299] Negotiation successful
0806 06:31:20.044461 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:31:20.044679 12331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:20.032925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:46083 (local address 127.7.43.67:37147)
0806 06:31:20.033115 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:20.033161 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:20.033633 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:20.033830 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:20.033833 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:20.034361 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:20.034547 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:20.034553 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:20.035165 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:20.035166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:20.035950 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:20.035952 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:20.036283 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:20.036284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:20.036372 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:20.040891 (+ 4519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:20.040899 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:20.042375 (+ 1476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:20.043602 (+ 1227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:20.043608 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:20.043609 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:20.043815 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:20.043962 (+ 147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:20.043962 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:20.043963 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:20.044046 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:20.044213 (+ 167us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:20.044216 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:20.044315 (+ 99us) client_negotiation.cc:769] Sending connection context
0806 06:31:20.044387 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 06:31:20.044492 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":78,"threads_started":1}
I20260806 06:31:20.045070 12329 heartbeater.cc:347] Connected to a master server at 127.7.43.126:46083
I20260806 06:31:20.045152 12329 heartbeater.cc:464] Registering TS with master...
I20260806 06:31:20.045311 12329 heartbeater.cc:511] Master 127.7.43.126:46083 requested a full tablet report, sending...
I20260806 06:31:20.045611 11666 ts_manager.cc:194] Registered new tserver with Master: 0fa8bf6518854f7bb78de37f49ac61cb (127.7.43.67:33755)
I20260806 06:31:20.045935 11666 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.67@KRBTEST.COM'} at 127.7.43.67:37147
I20260806 06:31:20.053061 7341 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 06:31:20 dist-test-slave-6zbq krb5kdc[11598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997880, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 06 06:31:20 dist-test-slave-6zbq krb5kdc[11598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997880, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:31:20.074407 11758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:20.068206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52740 (local address 127.7.43.126:46083)
0806 06:31:20.068249 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:20.068250 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:20.068363 (+ 113us) server_negotiation.cc:407] Connection header received
0806 06:31:20.068405 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:20.068406 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:20.068413 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:20.068461 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:20.068939 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:20.069416 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:20.070095 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:20.070274 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:20.070322 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:20.072195 (+ 1873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:20.072197 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:20.072197 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:20.072203 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:20.073320 (+ 1117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:20.073642 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:20.073643 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:20.073643 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:20.073700 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:20.073922 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:20.073922 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:20.073923 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:20.074048 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:20.074148 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:20.074242 (+ 94us) server_negotiation.cc:299] Negotiation successful
0806 06:31:20.074312 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260806 06:31:20.074734 11666 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:52740
I20260806 06:31:20.075071 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 11764
I20260806 06:31:20.092736 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 11954
I20260806 06:31:20.105827 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 12143
I20260806 06:31:20.112066 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 11620
2026-08-06T06:31:20Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1877 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:31:20 dist-test-slave-6zbq krb5kdc[12347](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:31:20 dist-test-slave-6zbq krb5kdc[12347](info): set up 2 sockets
Aug 06 06:31:20 dist-test-slave-6zbq krb5kdc[12347](info): commencing operation
krb5kdc: starting...
W20260806 06:31:21.216799 7341 mini_kdc.cc:121] Time spent starting KDC: real 1.081s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Aug 06 06:31:21 dist-test-slave-6zbq krb5kdc[12347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:31:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:31:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.43.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
I20260806 06:31:21.323120 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:33507
--webserver_interface=127.7.43.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:44215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.43.126:33507
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:31:21.401407 12375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:21.401605 12375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:21.401625 12375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:21.402902 12375 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:31:21.402948 12375 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:21.402961 12375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:21.402973 12375 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:31:21.402983 12375 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:31:21.404479 12375 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:44215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.43.126:33507
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:33507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.43.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12375
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:21.404670 12375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:21.404830 12375 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:21.409274 12386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:21.409410 12375 server_base.cc:1034] running on GCE node
W20260806 06:31:21.409289 12388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:21.409333 12385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:21.409715 12375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:21.410001 12375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:21.411139 12375 hybrid_clock.cc:648] HybridClock initialized: now 1785997881411098 us; error 54 us; skew 500 ppm
Aug 06 06:31:21 dist-test-slave-6zbq krb5kdc[12347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997881, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:21.413475 12375 init.cc:377] Logged in from keytab as kudu/127.7.43.126@KRBTEST.COM (short username kudu)
I20260806 06:31:21.413906 12375 webserver.cc:460] Webserver started at http://127.7.43.126:35831/ using document root <none> and password file <none>
I20260806 06:31:21.414112 12375 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:21.414160 12375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:21.414280 12375 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:21.415266 12375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/master-0/data/instance:
uuid: "bf2ad4024d74402590bfd15def14f009"
format_stamp: "Formatted at 2026-08-06 06:31:21 on dist-test-slave-6zbq"
server_key: "91fe741103ca73967165ae3f85d70479"
server_key_iv: "b47fef32deb60eddf7ee39b6c5bd1920"
server_key_version: "encryptionkey@0"
I20260806 06:31:21.415594 12375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/master-0/wal/instance:
uuid: "bf2ad4024d74402590bfd15def14f009"
format_stamp: "Formatted at 2026-08-06 06:31:21 on dist-test-slave-6zbq"
server_key: "91fe741103ca73967165ae3f85d70479"
server_key_iv: "b47fef32deb60eddf7ee39b6c5bd1920"
server_key_version: "encryptionkey@0"
I20260806 06:31:21.416795 12375 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:31:21.417423 12397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:21.417565 12375 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:31:21.417645 12375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "bf2ad4024d74402590bfd15def14f009"
format_stamp: "Formatted at 2026-08-06 06:31:21 on dist-test-slave-6zbq"
server_key: "91fe741103ca73967165ae3f85d70479"
server_key_iv: "b47fef32deb60eddf7ee39b6c5bd1920"
server_key_version: "encryptionkey@0"
I20260806 06:31:21.417706 12375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:21.433773 12375 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:21.435803 12375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:21.435935 12375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:21.439692 12375 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.126:33507
I20260806 06:31:21.439728 12469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.126:33507 every 8 connection(s)
I20260806 06:31:21.440234 12375 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:31:21.440851 12473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:21.443074 12473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf2ad4024d74402590bfd15def14f009: Bootstrap starting.
I20260806 06:31:21.443621 12473 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf2ad4024d74402590bfd15def14f009: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:21.443890 12473 log.cc:824] T 00000000000000000000000000000000 P bf2ad4024d74402590bfd15def14f009: Log is configured to *not* fsync() on all Append() calls
I20260806 06:31:21.444486 12473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf2ad4024d74402590bfd15def14f009: No bootstrap required, opened a new log
I20260806 06:31:21.445986 12473 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf2ad4024d74402590bfd15def14f009 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf2ad4024d74402590bfd15def14f009" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 33507 } }
I20260806 06:31:21.446117 12473 raft_consensus.cc:374] T 00000000000000000000000000000000 P bf2ad4024d74402590bfd15def14f009 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:21.446149 12473 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf2ad4024d74402590bfd15def14f009 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf2ad4024d74402590bfd15def14f009, State: Initialized, Role: FOLLOWER
I20260806 06:31:21.446288 12473 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf2ad4024d74402590bfd15def14f009 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf2ad4024d74402590bfd15def14f009" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 33507 } }
I20260806 06:31:21.446372 12473 raft_consensus.cc:388] T 00000000000000000000000000000000 P bf2ad4024d74402590bfd15def14f009 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:31:21.446410 12473 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf2ad4024d74402590bfd15def14f009 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:31:21.446444 12473 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf2ad4024d74402590bfd15def14f009 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:21.447192 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 12375
I20260806 06:31:21.447252 12473 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf2ad4024d74402590bfd15def14f009 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf2ad4024d74402590bfd15def14f009" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 33507 } }
I20260806 06:31:21.447281 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/master-0/wal/instance
I20260806 06:31:21.447352 12473 leader_election.cc:302] T 00000000000000000000000000000000 P bf2ad4024d74402590bfd15def14f009 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf2ad4024d74402590bfd15def14f009; no voters:
I20260806 06:31:21.447451 7341 external_mini_cluster.cc:1428] Setting key bbd45e3b29e059bc5b4f8415affd2e53
I20260806 06:31:21.447494 12473 leader_election.cc:288] T 00000000000000000000000000000000 P bf2ad4024d74402590bfd15def14f009 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:31:21.447520 12476 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf2ad4024d74402590bfd15def14f009 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:31:21.447692 12476 raft_consensus.cc:686] T 00000000000000000000000000000000 P bf2ad4024d74402590bfd15def14f009 [term 1 LEADER]: Becoming Leader. State: Replica: bf2ad4024d74402590bfd15def14f009, State: Running, Role: LEADER
I20260806 06:31:21.447808 12473 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf2ad4024d74402590bfd15def14f009 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:31:21.447788 12476 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf2ad4024d74402590bfd15def14f009 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf2ad4024d74402590bfd15def14f009" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 33507 } }
I20260806 06:31:21.448457 12476 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf2ad4024d74402590bfd15def14f009 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf2ad4024d74402590bfd15def14f009. Latest consensus state: current_term: 1 leader_uuid: "bf2ad4024d74402590bfd15def14f009" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf2ad4024d74402590bfd15def14f009" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 33507 } } }
I20260806 06:31:21.448511 12477 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf2ad4024d74402590bfd15def14f009 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bf2ad4024d74402590bfd15def14f009" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf2ad4024d74402590bfd15def14f009" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 33507 } } }
I20260806 06:31:21.448604 12476 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf2ad4024d74402590bfd15def14f009 [sys.catalog]: This master's current role is: LEADER
I20260806 06:31:21.448714 12477 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf2ad4024d74402590bfd15def14f009 [sys.catalog]: This master's current role is: LEADER
W20260806 06:31:21.449509 12501 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bf2ad4024d74402590bfd15def14f009: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:31:21.449551 12501 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:31:21.449635 12493 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:31:21.449803 12493 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:31:21.451062 12493 catalog_manager.cc:1347] Generated new cluster ID: 36310444fa194b9d8ddf36eb7bf060a2
I20260806 06:31:21.451110 12493 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:31:21 dist-test-slave-6zbq krb5kdc[12347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:31:21.458778 12483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:21.447972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35940 (local address 127.7.43.126:33507)
0806 06:31:21.448138 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:21.448147 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:21.448181 (+ 34us) server_negotiation.cc:407] Connection header received
0806 06:31:21.448290 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:21.448298 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:21.448354 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:21.449574 (+ 1220us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:21.449666 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:21.451036 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:21.452062 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:21.452714 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:21.452842 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:21.455421 (+ 2579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:21.455424 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:21.455425 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:21.455435 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:21.457155 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:21.457590 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:21.457593 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:21.457593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:21.457689 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:21.457894 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:21.457895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:21.457895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:21.458169 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:21.458310 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:21.458456 (+ 146us) server_negotiation.cc:299] Negotiation successful
0806 06:31:21.458582 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":72,"threads_started":1}
I20260806 06:31:21.460768 12493 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:31:21.461472 12493 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:31:21.466039 12493 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bf2ad4024d74402590bfd15def14f009: Generated new TSK 0
I20260806 06:31:21.466189 12493 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.43.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab.
Entry for principal kudu/127.7.43.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab.
I20260806 06:31:21.495200 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.65:0
--local_ip_for_outbound_sockets=127.7.43.65
--webserver_interface=127.7.43.65
--webserver_port=0
--tserver_master_addrs=127.7.43.126:33507
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab
--principal=kudu/127.7.43.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:44215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:31:21.578092 12509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:21.578289 12509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:21.578320 12509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:21.579488 12509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:21.579527 12509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:21.579571 12509 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.65
I20260806 06:31:21.581060 12509 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:44215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab
--principal=kudu/127.7.43.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.43.65
--webserver_port=0
--tserver_master_addrs=127.7.43.126:33507
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12509
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.65
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:21.581308 12509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:21.581501 12509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:21.586018 12514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:21.586125 12509 server_base.cc:1034] running on GCE node
W20260806 06:31:21.586016 12515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:21.586012 12520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:21.586448 12509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:21.586658 12509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:21.587801 12509 hybrid_clock.cc:648] HybridClock initialized: now 1785997881587788 us; error 26 us; skew 500 ppm
Aug 06 06:31:21 dist-test-slave-6zbq krb5kdc[12347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997881, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:21.590157 12509 init.cc:377] Logged in from keytab as kudu/127.7.43.65@KRBTEST.COM (short username kudu)
I20260806 06:31:21.590557 12509 webserver.cc:460] Webserver started at http://127.7.43.65:36237/ using document root <none> and password file <none>
I20260806 06:31:21.590749 12509 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:21.590790 12509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:21.590874 12509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:21.591784 12509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/ts-0/data/instance:
uuid: "ed5fc6351a254f12870d4c8800ecd567"
format_stamp: "Formatted at 2026-08-06 06:31:21 on dist-test-slave-6zbq"
server_key: "d330703d45a0d3402af93d4066132fe7"
server_key_iv: "9962740bed27364c917630ce3758f6c6"
server_key_version: "encryptionkey@0"
I20260806 06:31:21.592061 12509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/ts-0/wal/instance:
uuid: "ed5fc6351a254f12870d4c8800ecd567"
format_stamp: "Formatted at 2026-08-06 06:31:21 on dist-test-slave-6zbq"
server_key: "d330703d45a0d3402af93d4066132fe7"
server_key_iv: "9962740bed27364c917630ce3758f6c6"
server_key_version: "encryptionkey@0"
I20260806 06:31:21.593178 12509 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:31:21.593783 12531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:21.593935 12509 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:31:21.594010 12509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/ts-0/wal
uuid: "ed5fc6351a254f12870d4c8800ecd567"
format_stamp: "Formatted at 2026-08-06 06:31:21 on dist-test-slave-6zbq"
server_key: "d330703d45a0d3402af93d4066132fe7"
server_key_iv: "9962740bed27364c917630ce3758f6c6"
server_key_version: "encryptionkey@0"
I20260806 06:31:21.594064 12509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:21.607229 12509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:21.608748 12509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:21.608851 12509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:21.609004 12509 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:31:21.609228 12509 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:31:21.609257 12509 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:21.609277 12509 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:31:21.609293 12509 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:21.614557 12509 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.65:36581
I20260806 06:31:21.614624 12688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.65:36581 every 8 connection(s)
I20260806 06:31:21.614869 12509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
I20260806 06:31:21.619760 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 12509
I20260806 06:31:21.619845 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/ts-0/wal/instance
I20260806 06:31:21.619961 7341 external_mini_cluster.cc:1428] Setting key f91a5a176f8af96a00d3176a4c3905cd
Aug 06 06:31:21 dist-test-slave-6zbq krb5kdc[12347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997881, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.65@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:31:21.623412 12483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:21.615500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:55229 (local address 127.7.43.126:33507)
0806 06:31:21.615549 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:21.615551 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:21.615739 (+ 188us) server_negotiation.cc:407] Connection header received
0806 06:31:21.616556 (+ 817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:21.616557 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:21.616570 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:21.616617 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:21.617132 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:21.617617 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:21.618544 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:21.618738 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:21.618787 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:21.620866 (+ 2079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:21.620868 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:21.620868 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:21.620876 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:21.622191 (+ 1315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:21.622596 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:21.622597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:21.622598 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:21.622672 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:21.622878 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:21.622879 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:21.622879 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:21.623008 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:21.623072 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:21.623268 (+ 196us) server_negotiation.cc:299] Negotiation successful
0806 06:31:21.623349 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:31:21.623488 12694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:21.615481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33507 (local address 127.7.43.65:55229)
0806 06:31:21.615627 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:21.615658 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:21.616478 (+ 820us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:21.616674 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:21.616677 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:21.616896 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:21.617062 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:21.617066 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:21.617693 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:21.617693 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:21.618487 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:21.618490 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:21.618792 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:21.618793 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:21.618886 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:21.619305 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:21.619312 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:21.620784 (+ 1472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:21.622272 (+ 1488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:21.622279 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:21.622281 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:21.622532 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:21.622741 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:21.622741 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:21.622742 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:21.622832 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:21.623062 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:21.623064 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:21.623159 (+ 95us) client_negotiation.cc:769] Sending connection context
0806 06:31:21.623230 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 06:31:21.623331 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":79,"threads_started":1}
I20260806 06:31:21.624002 12689 heartbeater.cc:347] Connected to a master server at 127.7.43.126:33507
I20260806 06:31:21.624094 12689 heartbeater.cc:464] Registering TS with master...
I20260806 06:31:21.624262 12689 heartbeater.cc:511] Master 127.7.43.126:33507 requested a full tablet report, sending...
I20260806 06:31:21.624723 12422 ts_manager.cc:194] Registered new tserver with Master: ed5fc6351a254f12870d4c8800ecd567 (127.7.43.65:36581)
I20260806 06:31:21.625317 12422 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.65@KRBTEST.COM'} at 127.7.43.65:55229
WARNING: no policy specified for kudu/127.7.43.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab.
Entry for principal kudu/127.7.43.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab.
I20260806 06:31:21.644233 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.66:0
--local_ip_for_outbound_sockets=127.7.43.66
--webserver_interface=127.7.43.66
--webserver_port=0
--tserver_master_addrs=127.7.43.126:33507
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab
--principal=kudu/127.7.43.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:44215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:31:21.722822 12697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:21.723001 12697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:21.723022 12697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:21.724128 12697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:21.724167 12697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:21.724191 12697 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.66
I20260806 06:31:21.725618 12697 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:44215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab
--principal=kudu/127.7.43.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.43.66
--webserver_port=0
--tserver_master_addrs=127.7.43.126:33507
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12697
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.66
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:21.725826 12697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:21.726016 12697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:21.730677 12706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:21.730764 12697 server_base.cc:1034] running on GCE node
W20260806 06:31:21.730600 12710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:21.730595 12707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:21.731035 12697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:21.731249 12697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:21.732393 12697 hybrid_clock.cc:648] HybridClock initialized: now 1785997881732381 us; error 27 us; skew 500 ppm
Aug 06 06:31:21 dist-test-slave-6zbq krb5kdc[12347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997881, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:21.734683 12697 init.cc:377] Logged in from keytab as kudu/127.7.43.66@KRBTEST.COM (short username kudu)
I20260806 06:31:21.735030 12697 webserver.cc:460] Webserver started at http://127.7.43.66:42217/ using document root <none> and password file <none>
I20260806 06:31:21.735190 12697 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:21.735226 12697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:21.735311 12697 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:21.736209 12697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/ts-1/data/instance:
uuid: "f607954708ce4effa0a05a4860490b18"
format_stamp: "Formatted at 2026-08-06 06:31:21 on dist-test-slave-6zbq"
server_key: "3d271f9e262f0af969ff87deac49edc5"
server_key_iv: "2243e8a2f2b8bf16ba407e4a01636818"
server_key_version: "encryptionkey@0"
I20260806 06:31:21.736476 12697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/ts-1/wal/instance:
uuid: "f607954708ce4effa0a05a4860490b18"
format_stamp: "Formatted at 2026-08-06 06:31:21 on dist-test-slave-6zbq"
server_key: "3d271f9e262f0af969ff87deac49edc5"
server_key_iv: "2243e8a2f2b8bf16ba407e4a01636818"
server_key_version: "encryptionkey@0"
I20260806 06:31:21.737571 12697 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:31:21.738196 12720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:21.738334 12697 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:31:21.738451 12697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/ts-1/wal
uuid: "f607954708ce4effa0a05a4860490b18"
format_stamp: "Formatted at 2026-08-06 06:31:21 on dist-test-slave-6zbq"
server_key: "3d271f9e262f0af969ff87deac49edc5"
server_key_iv: "2243e8a2f2b8bf16ba407e4a01636818"
server_key_version: "encryptionkey@0"
I20260806 06:31:21.738500 12697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:21.753852 12697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:21.755836 12697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:21.755949 12697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:21.756151 12697 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:31:21.756421 12697 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:31:21.756454 12697 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:21.756474 12697 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:31:21.756486 12697 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:21.761880 12697 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.66:33275
I20260806 06:31:21.761950 12877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.66:33275 every 8 connection(s)
I20260806 06:31:21.762184 12697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
Aug 06 06:31:21 dist-test-slave-6zbq krb5kdc[12347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997881, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.66@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:31:21.768374 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 12697
I20260806 06:31:21.768476 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/ts-1/wal/instance
I20260806 06:31:21.768661 7341 external_mini_cluster.cc:1428] Setting key 170d35b40c0520d343d5adf48663c7ef
I20260806 06:31:21.770687 12483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:21.762849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:48053 (local address 127.7.43.126:33507)
0806 06:31:21.762913 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:21.762915 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:21.763083 (+ 168us) server_negotiation.cc:407] Connection header received
0806 06:31:21.763934 (+ 851us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:21.763935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:21.763944 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:21.763989 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:21.764529 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:21.765007 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:21.765923 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:21.766110 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:21.766164 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:21.768003 (+ 1839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:21.768005 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:21.768005 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:21.768012 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:21.769433 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:21.769789 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:21.769790 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:21.769791 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:21.769848 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:21.770054 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:21.770055 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:21.770056 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:21.770171 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:21.770226 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:21.770516 (+ 290us) server_negotiation.cc:299] Negotiation successful
0806 06:31:21.770600 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 06:31:21.770829 12885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:21.762806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33507 (local address 127.7.43.66:48053)
0806 06:31:21.762971 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:21.763009 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:21.763852 (+ 843us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:21.764049 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:21.764053 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:21.764287 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:21.764468 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:21.764472 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:21.765065 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:21.765066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:21.765844 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:21.765847 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:21.766175 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:21.766176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:21.766269 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:21.766674 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:21.766682 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:21.767921 (+ 1239us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:21.769501 (+ 1580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:21.769507 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:21.769509 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:21.769723 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:21.769900 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:21.769901 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:21.769901 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:21.769999 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:21.770226 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:21.770233 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:21.770381 (+ 148us) client_negotiation.cc:769] Sending connection context
0806 06:31:21.770475 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 06:31:21.770628 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":93,"threads_started":1}
I20260806 06:31:21.771324 12878 heartbeater.cc:347] Connected to a master server at 127.7.43.126:33507
I20260806 06:31:21.771405 12878 heartbeater.cc:464] Registering TS with master...
I20260806 06:31:21.771560 12878 heartbeater.cc:511] Master 127.7.43.126:33507 requested a full tablet report, sending...
I20260806 06:31:21.771921 12422 ts_manager.cc:194] Registered new tserver with Master: f607954708ce4effa0a05a4860490b18 (127.7.43.66:33275)
I20260806 06:31:21.772266 12422 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.66@KRBTEST.COM'} at 127.7.43.66:48053
WARNING: no policy specified for kudu/127.7.43.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab.
Entry for principal kudu/127.7.43.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab.
I20260806 06:31:21.790812 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.67:0
--local_ip_for_outbound_sockets=127.7.43.67
--webserver_interface=127.7.43.67
--webserver_port=0
--tserver_master_addrs=127.7.43.126:33507
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab
--principal=kudu/127.7.43.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:44215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:31:21.868799 12888 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:21.868983 12888 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:21.869004 12888 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:21.870111 12888 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:21.870150 12888 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:21.870186 12888 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.67
I20260806 06:31:21.871675 12888 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:44215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab
--principal=kudu/127.7.43.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.43.67
--webserver_port=0
--tserver_master_addrs=127.7.43.126:33507
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12888
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.67
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:21.871887 12888 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:21.872080 12888 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:21.876659 12896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:21.876731 12888 server_base.cc:1034] running on GCE node
W20260806 06:31:21.876695 12895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:21.876885 12900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:21.877095 12888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:21.877285 12888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:21.878391 12888 hybrid_clock.cc:648] HybridClock initialized: now 1785997881878377 us; error 28 us; skew 500 ppm
Aug 06 06:31:21 dist-test-slave-6zbq krb5kdc[12347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997881, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:21.880661 12888 init.cc:377] Logged in from keytab as kudu/127.7.43.67@KRBTEST.COM (short username kudu)
I20260806 06:31:21.881040 12888 webserver.cc:460] Webserver started at http://127.7.43.67:45719/ using document root <none> and password file <none>
I20260806 06:31:21.881242 12888 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:21.881289 12888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:21.881407 12888 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:21.882397 12888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/ts-2/data/instance:
uuid: "57f94a7f2d9a4f3ab36d8713d5623097"
format_stamp: "Formatted at 2026-08-06 06:31:21 on dist-test-slave-6zbq"
server_key: "fbcbece747f0ca42e043ec42280c4212"
server_key_iv: "dc3426679948564fe1db0f62ee09aaa4"
server_key_version: "encryptionkey@0"
I20260806 06:31:21.882721 12888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/ts-2/wal/instance:
uuid: "57f94a7f2d9a4f3ab36d8713d5623097"
format_stamp: "Formatted at 2026-08-06 06:31:21 on dist-test-slave-6zbq"
server_key: "fbcbece747f0ca42e043ec42280c4212"
server_key_iv: "dc3426679948564fe1db0f62ee09aaa4"
server_key_version: "encryptionkey@0"
I20260806 06:31:21.883895 12888 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:31:21.884527 12910 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:21.884671 12888 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:31:21.884759 12888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/ts-2/wal
uuid: "57f94a7f2d9a4f3ab36d8713d5623097"
format_stamp: "Formatted at 2026-08-06 06:31:21 on dist-test-slave-6zbq"
server_key: "fbcbece747f0ca42e043ec42280c4212"
server_key_iv: "dc3426679948564fe1db0f62ee09aaa4"
server_key_version: "encryptionkey@0"
I20260806 06:31:21.884817 12888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:21.894212 12888 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:21.895793 12888 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:21.895891 12888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:21.896049 12888 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:31:21.896306 12888 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:31:21.896338 12888 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:21.896363 12888 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:31:21.896381 12888 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:21.901818 12888 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.67:44389
I20260806 06:31:21.901878 13058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.67:44389 every 8 connection(s)
I20260806 06:31:21.902122 12888 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
I20260806 06:31:21.904623 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 12888
I20260806 06:31:21.904702 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785997861828271-7341-0/minicluster-data/ts-2/wal/instance
I20260806 06:31:21.904858 7341 external_mini_cluster.cc:1428] Setting key d1e1c6cd6ddae068ca69c66802266838
Aug 06 06:31:21 dist-test-slave-6zbq krb5kdc[12347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997881, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.67@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:31:21.910537 12483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:21.902751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:34999 (local address 127.7.43.126:33507)
0806 06:31:21.902811 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:21.902813 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:21.902965 (+ 152us) server_negotiation.cc:407] Connection header received
0806 06:31:21.903830 (+ 865us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:21.903831 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:21.903845 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:21.903892 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:21.904462 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:21.905023 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:21.906190 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:21.906461 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:21.906520 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:21.908273 (+ 1753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:21.908274 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:21.908275 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:21.908281 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:21.909427 (+ 1146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:21.909753 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:21.909754 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:21.909754 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:21.909798 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:21.910003 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:21.910004 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:21.910004 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:21.910120 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:21.910179 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:21.910412 (+ 233us) server_negotiation.cc:299] Negotiation successful
0806 06:31:21.910474 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 06:31:21.910705 13063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:21.902699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33507 (local address 127.7.43.67:34999)
0806 06:31:21.902858 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:21.902896 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:21.903730 (+ 834us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:21.903968 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:21.903972 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:21.904198 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:21.904376 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:21.904381 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:21.905162 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:21.905163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:21.906098 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:21.906101 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:21.906522 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:21.906523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:21.906614 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:21.906970 (+ 356us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:21.906977 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:21.908193 (+ 1216us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:21.909499 (+ 1306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:21.909504 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:21.909506 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:21.909698 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:21.909841 (+ 143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:21.909842 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:21.909842 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:21.909924 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:21.910182 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:21.910185 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:21.910283 (+ 98us) client_negotiation.cc:769] Sending connection context
0806 06:31:21.910397 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 06:31:21.910515 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":81,"threads_started":1}
I20260806 06:31:21.911130 13059 heartbeater.cc:347] Connected to a master server at 127.7.43.126:33507
I20260806 06:31:21.911221 13059 heartbeater.cc:464] Registering TS with master...
I20260806 06:31:21.911381 13059 heartbeater.cc:511] Master 127.7.43.126:33507 requested a full tablet report, sending...
I20260806 06:31:21.911708 12422 ts_manager.cc:194] Registered new tserver with Master: 57f94a7f2d9a4f3ab36d8713d5623097 (127.7.43.67:44389)
I20260806 06:31:21.912053 12422 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.67@KRBTEST.COM'} at 127.7.43.67:34999
I20260806 06:31:21.916219 7341 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 06:31:21 dist-test-slave-6zbq krb5kdc[12347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 06:31:21 dist-test-slave-6zbq krb5kdc[12347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.43.65@KRBTEST.COM
I20260806 06:31:21.938198 13068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:21.931228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41724 (local address 127.7.43.65:36581)
0806 06:31:21.931392 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:21.931396 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:21.931404 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:31:21.931427 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:21.931429 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:21.931441 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:21.931504 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:21.932055 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:21.932782 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:21.933498 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:21.933810 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:21.933870 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:21.935960 (+ 2090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:21.935968 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:21.935968 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:21.935975 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:21.937273 (+ 1298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:21.937565 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:21.937567 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:21.937568 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:21.937626 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:21.937759 (+ 133us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:21.937760 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:21.937760 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:21.937871 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:21.937961 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:21.938016 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 06:31:21.938077 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":78,"threads_started":1}
Aug 06 06:31:21 dist-test-slave-6zbq krb5kdc[12347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:31:21.945254 12483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:21.939438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35954 (local address 127.7.43.126:33507)
0806 06:31:21.939498 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:21.939500 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:21.939508 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:31:21.939531 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:21.939532 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:21.939541 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:21.939586 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:21.940024 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:21.940486 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:21.941167 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:21.941361 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:21.941413 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:21.943242 (+ 1829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:21.943244 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:21.943244 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:21.943251 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:21.944258 (+ 1007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:21.944555 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:21.944556 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:21.944556 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:21.944608 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:21.944761 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:21.944761 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:21.944762 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:21.944896 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:21.944980 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:21.945101 (+ 121us) server_negotiation.cc:299] Negotiation successful
0806 06:31:21.945170 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
W20260806 06:31:21.945457 12422 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35954
I20260806 06:31:21.945744 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 12509
I20260806 06:31:21.963514 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 12697
I20260806 06:31:21.971683 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 12888
I20260806 06:31:21.978628 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 12375
2026-08-06T06:31:21Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1862 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:31:22 dist-test-slave-6zbq krb5kdc[13075](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:31:22 dist-test-slave-6zbq krb5kdc[13075](info): set up 2 sockets
Aug 06 06:31:22 dist-test-slave-6zbq krb5kdc[13075](info): commencing operation
krb5kdc: starting...
W20260806 06:31:23.075592 7341 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Aug 06 06:31:23 dist-test-slave-6zbq krb5kdc[13075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997883, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:31:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:31:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.43.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
I20260806 06:31:23.178781 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:34557
--webserver_interface=127.7.43.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:33819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.43.126:34557
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:31:23.256294 13098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:23.256481 13098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:23.256511 13098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:23.257617 13098 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:31:23.257656 13098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:23.257675 13098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:23.257690 13098 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:31:23.257704 13098 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:31:23.259176 13098 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:33819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.43.126:34557
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:34557
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.43.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13098
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:23.259411 13098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:23.259608 13098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:23.264068 13104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:23.264122 13105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:23.264082 13108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:23.264309 13098 server_base.cc:1034] running on GCE node
I20260806 06:31:23.264530 13098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:23.264806 13098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:23.265942 13098 hybrid_clock.cc:648] HybridClock initialized: now 1785997883265929 us; error 28 us; skew 500 ppm
Aug 06 06:31:23 dist-test-slave-6zbq krb5kdc[13075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997883, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:23.268213 13098 init.cc:377] Logged in from keytab as kudu/127.7.43.126@KRBTEST.COM (short username kudu)
I20260806 06:31:23.268504 13098 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:23.268553 13098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:23.268672 13098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:23.269616 13098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/master-0/data/instance:
uuid: "d2ecafe8d84c4c05ba95ea992b1b8f26"
format_stamp: "Formatted at 2026-08-06 06:31:23 on dist-test-slave-6zbq"
server_key: "40e9083221164f0e83a95c69fd12a081"
server_key_iv: "11bc321f1f06a572973fa6e7dbe61f16"
server_key_version: "encryptionkey@0"
I20260806 06:31:23.269930 13098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/master-0/wal/instance:
uuid: "d2ecafe8d84c4c05ba95ea992b1b8f26"
format_stamp: "Formatted at 2026-08-06 06:31:23 on dist-test-slave-6zbq"
server_key: "40e9083221164f0e83a95c69fd12a081"
server_key_iv: "11bc321f1f06a572973fa6e7dbe61f16"
server_key_version: "encryptionkey@0"
I20260806 06:31:23.271116 13098 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:31:23.271744 13117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:23.271894 13098 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:31:23.271973 13098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "d2ecafe8d84c4c05ba95ea992b1b8f26"
format_stamp: "Formatted at 2026-08-06 06:31:23 on dist-test-slave-6zbq"
server_key: "40e9083221164f0e83a95c69fd12a081"
server_key_iv: "11bc321f1f06a572973fa6e7dbe61f16"
server_key_version: "encryptionkey@0"
I20260806 06:31:23.272032 13098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:23.298316 13098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:23.299948 13098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:23.300048 13098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:23.303793 13098 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.126:34557
I20260806 06:31:23.303872 13193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.126:34557 every 8 connection(s)
I20260806 06:31:23.304123 13098 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:31:23.305186 13195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:23.307305 13195 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2ecafe8d84c4c05ba95ea992b1b8f26: Bootstrap starting.
I20260806 06:31:23.307848 13195 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d2ecafe8d84c4c05ba95ea992b1b8f26: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:23.308104 13195 log.cc:824] T 00000000000000000000000000000000 P d2ecafe8d84c4c05ba95ea992b1b8f26: Log is configured to *not* fsync() on all Append() calls
I20260806 06:31:23.308702 13195 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2ecafe8d84c4c05ba95ea992b1b8f26: No bootstrap required, opened a new log
I20260806 06:31:23.310205 13195 raft_consensus.cc:348] T 00000000000000000000000000000000 P d2ecafe8d84c4c05ba95ea992b1b8f26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2ecafe8d84c4c05ba95ea992b1b8f26" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 34557 } }
I20260806 06:31:23.310333 13195 raft_consensus.cc:374] T 00000000000000000000000000000000 P d2ecafe8d84c4c05ba95ea992b1b8f26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:23.310392 13195 raft_consensus.cc:729] T 00000000000000000000000000000000 P d2ecafe8d84c4c05ba95ea992b1b8f26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2ecafe8d84c4c05ba95ea992b1b8f26, State: Initialized, Role: FOLLOWER
I20260806 06:31:23.310529 13195 consensus_queue.cc:260] T 00000000000000000000000000000000 P d2ecafe8d84c4c05ba95ea992b1b8f26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2ecafe8d84c4c05ba95ea992b1b8f26" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 34557 } }
I20260806 06:31:23.310639 13195 raft_consensus.cc:388] T 00000000000000000000000000000000 P d2ecafe8d84c4c05ba95ea992b1b8f26 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:31:23.310680 13195 raft_consensus.cc:482] T 00000000000000000000000000000000 P d2ecafe8d84c4c05ba95ea992b1b8f26 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:31:23.310727 13195 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d2ecafe8d84c4c05ba95ea992b1b8f26 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:23.311474 13195 raft_consensus.cc:504] T 00000000000000000000000000000000 P d2ecafe8d84c4c05ba95ea992b1b8f26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2ecafe8d84c4c05ba95ea992b1b8f26" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 34557 } }
I20260806 06:31:23.311563 13195 leader_election.cc:302] T 00000000000000000000000000000000 P d2ecafe8d84c4c05ba95ea992b1b8f26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d2ecafe8d84c4c05ba95ea992b1b8f26; no voters:
I20260806 06:31:23.311709 13195 leader_election.cc:288] T 00000000000000000000000000000000 P d2ecafe8d84c4c05ba95ea992b1b8f26 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:31:23.311748 13199 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d2ecafe8d84c4c05ba95ea992b1b8f26 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:31:23.311878 13199 raft_consensus.cc:686] T 00000000000000000000000000000000 P d2ecafe8d84c4c05ba95ea992b1b8f26 [term 1 LEADER]: Becoming Leader. State: Replica: d2ecafe8d84c4c05ba95ea992b1b8f26, State: Running, Role: LEADER
I20260806 06:31:23.311939 13195 sys_catalog.cc:564] T 00000000000000000000000000000000 P d2ecafe8d84c4c05ba95ea992b1b8f26 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:31:23.311968 13199 consensus_queue.cc:237] T 00000000000000000000000000000000 P d2ecafe8d84c4c05ba95ea992b1b8f26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2ecafe8d84c4c05ba95ea992b1b8f26" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 34557 } }
I20260806 06:31:23.312295 13201 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2ecafe8d84c4c05ba95ea992b1b8f26 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d2ecafe8d84c4c05ba95ea992b1b8f26. Latest consensus state: current_term: 1 leader_uuid: "d2ecafe8d84c4c05ba95ea992b1b8f26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2ecafe8d84c4c05ba95ea992b1b8f26" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 34557 } } }
I20260806 06:31:23.312386 13201 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2ecafe8d84c4c05ba95ea992b1b8f26 [sys.catalog]: This master's current role is: LEADER
I20260806 06:31:23.312290 13200 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2ecafe8d84c4c05ba95ea992b1b8f26 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d2ecafe8d84c4c05ba95ea992b1b8f26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2ecafe8d84c4c05ba95ea992b1b8f26" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 34557 } } }
I20260806 06:31:23.312520 13200 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2ecafe8d84c4c05ba95ea992b1b8f26 [sys.catalog]: This master's current role is: LEADER
I20260806 06:31:23.312601 13209 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:31:23.312711 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 13098
I20260806 06:31:23.312815 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/master-0/wal/instance
I20260806 06:31:23.312995 7341 external_mini_cluster.cc:1428] Setting key 6ac322180b3c6524a9837643d7388aab
I20260806 06:31:23.313062 13209 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:31:23.315105 13209 catalog_manager.cc:1347] Generated new cluster ID: 08cc71d2fd6a463c898d11cc4430abd1
I20260806 06:31:23.315155 13209 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:31:23 dist-test-slave-6zbq krb5kdc[13075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997883, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:31:23.323987 13223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:23.313576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50240 (local address 127.7.43.126:34557)
0806 06:31:23.313727 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:23.313735 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:23.314007 (+ 272us) server_negotiation.cc:407] Connection header received
0806 06:31:23.314129 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:23.314135 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:23.314186 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:23.314311 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:23.315041 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:23.316474 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:23.317472 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:23.318087 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:23.318210 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:23.320690 (+ 2480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:23.320694 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:23.320695 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:23.320706 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:23.322506 (+ 1800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:23.322887 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:23.322890 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:23.322890 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:23.322983 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:23.323178 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:23.323178 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:23.323178 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:23.323436 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:23.323572 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:23.323659 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 06:31:23.323782 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":83,"threads_started":1}
I20260806 06:31:23.335827 13209 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:31:23.336508 13209 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:31:23.340559 13209 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d2ecafe8d84c4c05ba95ea992b1b8f26: Generated new TSK 0
I20260806 06:31:23.340708 13209 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.43.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab.
Entry for principal kudu/127.7.43.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab.
I20260806 06:31:23.366453 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.65:0
--local_ip_for_outbound_sockets=127.7.43.65
--webserver_interface=127.7.43.65
--webserver_port=0
--tserver_master_addrs=127.7.43.126:34557
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab
--principal=kudu/127.7.43.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:33819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:31:23.443939 13235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:23.444119 13235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:23.444149 13235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:23.445240 13235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:23.445277 13235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:23.445312 13235 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.65
I20260806 06:31:23.446763 13235 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:33819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab
--principal=kudu/127.7.43.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.43.65
--webserver_port=0
--tserver_master_addrs=127.7.43.126:34557
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13235
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.65
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:23.446981 13235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:23.447163 13235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:23.451669 13248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:23.451695 13245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:23.451690 13244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:23.451870 13235 server_base.cc:1034] running on GCE node
I20260806 06:31:23.452064 13235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:23.452265 13235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:23.453411 13235 hybrid_clock.cc:648] HybridClock initialized: now 1785997883453394 us; error 28 us; skew 500 ppm
Aug 06 06:31:23 dist-test-slave-6zbq krb5kdc[13075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997883, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:23.455782 13235 init.cc:377] Logged in from keytab as kudu/127.7.43.65@KRBTEST.COM (short username kudu)
I20260806 06:31:23.456086 13235 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:23.456135 13235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:23.456254 13235 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:23.457165 13235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/ts-0/data/instance:
uuid: "4d3751adc69240078920344d64b9058c"
format_stamp: "Formatted at 2026-08-06 06:31:23 on dist-test-slave-6zbq"
server_key: "7df58699d92d13eaea26a1e58764ae19"
server_key_iv: "a2b6aba8158dfc29f0af345d156aa2c8"
server_key_version: "encryptionkey@0"
I20260806 06:31:23.457471 13235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/ts-0/wal/instance:
uuid: "4d3751adc69240078920344d64b9058c"
format_stamp: "Formatted at 2026-08-06 06:31:23 on dist-test-slave-6zbq"
server_key: "7df58699d92d13eaea26a1e58764ae19"
server_key_iv: "a2b6aba8158dfc29f0af345d156aa2c8"
server_key_version: "encryptionkey@0"
I20260806 06:31:23.458586 13235 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:31:23.459206 13258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:23.459350 13235 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:31:23.459429 13235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/ts-0/wal
uuid: "4d3751adc69240078920344d64b9058c"
format_stamp: "Formatted at 2026-08-06 06:31:23 on dist-test-slave-6zbq"
server_key: "7df58699d92d13eaea26a1e58764ae19"
server_key_iv: "a2b6aba8158dfc29f0af345d156aa2c8"
server_key_version: "encryptionkey@0"
I20260806 06:31:23.459486 13235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:23.483160 13235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:23.484797 13235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:23.484912 13235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:23.485100 13235 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:31:23.485375 13235 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:31:23.485414 13235 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:23.485440 13235 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:31:23.485457 13235 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:23.490937 13235 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.65:37049
I20260806 06:31:23.490985 13419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.65:37049 every 8 connection(s)
I20260806 06:31:23.491215 13235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
Aug 06 06:31:23 dist-test-slave-6zbq krb5kdc[13075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997883, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.65@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:31:23.499140 13223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:23.491788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:42111 (local address 127.7.43.126:34557)
0806 06:31:23.491860 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:23.491862 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:23.492033 (+ 171us) server_negotiation.cc:407] Connection header received
0806 06:31:23.492438 (+ 405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:23.492439 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:23.492452 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:23.492498 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:23.493442 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:23.493937 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:23.494905 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:23.495087 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:23.495131 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:23.496953 (+ 1822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:23.496954 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:23.496955 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:23.496961 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:23.498145 (+ 1184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:23.498486 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:23.498487 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:23.498487 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:23.498534 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:23.498700 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:23.498701 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:23.498701 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:23.498798 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:23.498840 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:23.499022 (+ 182us) server_negotiation.cc:299] Negotiation successful
0806 06:31:23.499078 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 06:31:23.499296 13423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:23.491781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:34557 (local address 127.7.43.65:42111)
0806 06:31:23.491925 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:23.491958 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:23.492364 (+ 406us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:23.492546 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:23.492549 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:23.492785 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:23.493332 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:23.493338 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:23.494006 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:23.494006 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:23.494842 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:23.494845 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:23.495138 (+ 293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:23.495139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:23.495225 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:23.495574 (+ 349us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:23.495582 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:23.496859 (+ 1277us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:23.498208 (+ 1349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:23.498210 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:23.498212 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:23.498438 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:23.498581 (+ 143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:23.498581 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:23.498582 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:23.498665 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:23.498837 (+ 172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:23.498840 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:23.498937 (+ 97us) client_negotiation.cc:769] Sending connection context
0806 06:31:23.499009 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 06:31:23.499111 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":81,"threads_started":1}
I20260806 06:31:23.499729 13421 heartbeater.cc:347] Connected to a master server at 127.7.43.126:34557
I20260806 06:31:23.499809 13421 heartbeater.cc:464] Registering TS with master...
I20260806 06:31:23.499938 13421 heartbeater.cc:511] Master 127.7.43.126:34557 requested a full tablet report, sending...
I20260806 06:31:23.500375 13140 ts_manager.cc:194] Registered new tserver with Master: 4d3751adc69240078920344d64b9058c (127.7.43.65:37049)
I20260806 06:31:23.500813 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 13235
I20260806 06:31:23.500903 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/ts-0/wal/instance
I20260806 06:31:23.500974 13140 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.65@KRBTEST.COM'} at 127.7.43.65:42111
I20260806 06:31:23.501065 7341 external_mini_cluster.cc:1428] Setting key 57dfacb3f30739c0c00c8bcfad4e8433
WARNING: no policy specified for kudu/127.7.43.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab.
Entry for principal kudu/127.7.43.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab.
I20260806 06:31:23.522716 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.66:0
--local_ip_for_outbound_sockets=127.7.43.66
--webserver_interface=127.7.43.66
--webserver_port=0
--tserver_master_addrs=127.7.43.126:34557
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab
--principal=kudu/127.7.43.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:33819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:31:23.599604 13427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:23.599785 13427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:23.599820 13427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:23.600889 13427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:23.600919 13427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:23.600942 13427 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.66
I20260806 06:31:23.602332 13427 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:33819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab
--principal=kudu/127.7.43.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.43.66
--webserver_port=0
--tserver_master_addrs=127.7.43.126:34557
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13427
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.66
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:23.602514 13427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:23.602669 13427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:23.607118 13433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:23.607151 13438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:23.607163 13427 server_base.cc:1034] running on GCE node
W20260806 06:31:23.607163 13435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:23.607519 13427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:23.607698 13427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:23.608830 13427 hybrid_clock.cc:648] HybridClock initialized: now 1785997883608813 us; error 31 us; skew 500 ppm
Aug 06 06:31:23 dist-test-slave-6zbq krb5kdc[13075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997883, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:23.611035 13427 init.cc:377] Logged in from keytab as kudu/127.7.43.66@KRBTEST.COM (short username kudu)
I20260806 06:31:23.611341 13427 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:23.611388 13427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:23.611472 13427 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:23.612352 13427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/ts-1/data/instance:
uuid: "148ba296c40b4a9baca9e19364a7d31e"
format_stamp: "Formatted at 2026-08-06 06:31:23 on dist-test-slave-6zbq"
server_key: "ee76a9bbbc01b0933a3dfa6929d9eac4"
server_key_iv: "1e17855fe11c1424ee61d00e41766fc1"
server_key_version: "encryptionkey@0"
I20260806 06:31:23.612699 13427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/ts-1/wal/instance:
uuid: "148ba296c40b4a9baca9e19364a7d31e"
format_stamp: "Formatted at 2026-08-06 06:31:23 on dist-test-slave-6zbq"
server_key: "ee76a9bbbc01b0933a3dfa6929d9eac4"
server_key_iv: "1e17855fe11c1424ee61d00e41766fc1"
server_key_version: "encryptionkey@0"
I20260806 06:31:23.613794 13427 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:31:23.614405 13446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:23.614545 13427 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:23.614617 13427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/ts-1/wal
uuid: "148ba296c40b4a9baca9e19364a7d31e"
format_stamp: "Formatted at 2026-08-06 06:31:23 on dist-test-slave-6zbq"
server_key: "ee76a9bbbc01b0933a3dfa6929d9eac4"
server_key_iv: "1e17855fe11c1424ee61d00e41766fc1"
server_key_version: "encryptionkey@0"
I20260806 06:31:23.614679 13427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:23.636440 13427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:23.638324 13427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:23.638473 13427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:23.638666 13427 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:31:23.638933 13427 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:31:23.638964 13427 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:23.638983 13427 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:31:23.639007 13427 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:23.644415 13427 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.66:46449
I20260806 06:31:23.644474 13612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.66:46449 every 8 connection(s)
I20260806 06:31:23.644670 13427 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
I20260806 06:31:23.646672 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 13427
I20260806 06:31:23.646775 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/ts-1/wal/instance
I20260806 06:31:23.646952 7341 external_mini_cluster.cc:1428] Setting key c45c8391962b9ab91017d04303f3c0ee
Aug 06 06:31:23 dist-test-slave-6zbq krb5kdc[13075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997883, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.66@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:31:23.653410 13223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:23.645227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:42771 (local address 127.7.43.126:34557)
0806 06:31:23.645280 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:23.645282 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:23.645475 (+ 193us) server_negotiation.cc:407] Connection header received
0806 06:31:23.645858 (+ 383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:23.645859 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:23.645869 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:23.645916 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:23.646853 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:23.647511 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:23.648776 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:23.648960 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:23.649011 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:23.650958 (+ 1947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:23.650959 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:23.650959 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:23.650967 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:23.652230 (+ 1263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:23.652573 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:23.652574 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:23.652574 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:23.652627 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:23.652834 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:23.652835 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:23.652836 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:23.652943 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:23.652999 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:23.653238 (+ 239us) server_negotiation.cc:299] Negotiation successful
0806 06:31:23.653323 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 06:31:23.653553 13615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:23.645222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:34557 (local address 127.7.43.66:42771)
0806 06:31:23.645368 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:23.645398 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:23.645785 (+ 387us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:23.645962 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:23.645965 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:23.646181 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:23.646706 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:23.646714 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:23.647586 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:23.647587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:23.648696 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:23.648699 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:23.649029 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:23.649030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:23.649162 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:23.649590 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:23.649598 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:23.650874 (+ 1276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:23.652303 (+ 1429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:23.652309 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:23.652311 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:23.652509 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:23.652680 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:23.652681 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:23.652681 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:23.652779 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:23.652997 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:23.653003 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:23.653126 (+ 123us) client_negotiation.cc:769] Sending connection context
0806 06:31:23.653209 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 06:31:23.653360 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":77,"threads_started":1}
I20260806 06:31:23.654117 13613 heartbeater.cc:347] Connected to a master server at 127.7.43.126:34557
I20260806 06:31:23.654191 13613 heartbeater.cc:464] Registering TS with master...
I20260806 06:31:23.654309 13613 heartbeater.cc:511] Master 127.7.43.126:34557 requested a full tablet report, sending...
I20260806 06:31:23.654685 13140 ts_manager.cc:194] Registered new tserver with Master: 148ba296c40b4a9baca9e19364a7d31e (127.7.43.66:46449)
I20260806 06:31:23.655169 13140 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.66@KRBTEST.COM'} at 127.7.43.66:42771
WARNING: no policy specified for kudu/127.7.43.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab.
Entry for principal kudu/127.7.43.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab.
I20260806 06:31:23.669842 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.67:0
--local_ip_for_outbound_sockets=127.7.43.67
--webserver_interface=127.7.43.67
--webserver_port=0
--tserver_master_addrs=127.7.43.126:34557
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab
--principal=kudu/127.7.43.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:33819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:31:23.747027 13620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:23.747198 13620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:23.747220 13620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:23.748317 13620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:23.748355 13620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:23.748384 13620 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.67
I20260806 06:31:23.749822 13620 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:33819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab
--principal=kudu/127.7.43.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.43.67
--webserver_port=0
--tserver_master_addrs=127.7.43.126:34557
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13620
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.67
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:23.750031 13620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:23.750214 13620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:23.754878 13632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:23.754873 13628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:23.754992 13620 server_base.cc:1034] running on GCE node
W20260806 06:31:23.754873 13626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:23.755287 13620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:23.755504 13620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:23.756642 13620 hybrid_clock.cc:648] HybridClock initialized: now 1785997883756628 us; error 27 us; skew 500 ppm
Aug 06 06:31:23 dist-test-slave-6zbq krb5kdc[13075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997883, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:23.758914 13620 init.cc:377] Logged in from keytab as kudu/127.7.43.67@KRBTEST.COM (short username kudu)
I20260806 06:31:23.760258 13620 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:23.760306 13620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:23.760419 13620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:23.761371 13620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/ts-2/data/instance:
uuid: "795001fb9ee24ab38ac90a363195383d"
format_stamp: "Formatted at 2026-08-06 06:31:23 on dist-test-slave-6zbq"
server_key: "4cb6991d6b1ee701ef2f6fb2c65e0b8d"
server_key_iv: "13c09d7a0cb3f50e46c01071d47b4983"
server_key_version: "encryptionkey@0"
I20260806 06:31:23.761675 13620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/ts-2/wal/instance:
uuid: "795001fb9ee24ab38ac90a363195383d"
format_stamp: "Formatted at 2026-08-06 06:31:23 on dist-test-slave-6zbq"
server_key: "4cb6991d6b1ee701ef2f6fb2c65e0b8d"
server_key_iv: "13c09d7a0cb3f50e46c01071d47b4983"
server_key_version: "encryptionkey@0"
I20260806 06:31:23.762873 13620 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:31:23.763530 13639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:23.763686 13620 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:31:23.763763 13620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/ts-2/wal
uuid: "795001fb9ee24ab38ac90a363195383d"
format_stamp: "Formatted at 2026-08-06 06:31:23 on dist-test-slave-6zbq"
server_key: "4cb6991d6b1ee701ef2f6fb2c65e0b8d"
server_key_iv: "13c09d7a0cb3f50e46c01071d47b4983"
server_key_version: "encryptionkey@0"
I20260806 06:31:23.763831 13620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:23.773026 13620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:23.774928 13620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:23.775046 13620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:23.775241 13620 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:31:23.775514 13620 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:31:23.775547 13620 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:23.775566 13620 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:31:23.775589 13620 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:23.780979 13620 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.67:42559
I20260806 06:31:23.781013 13808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.67:42559 every 8 connection(s)
I20260806 06:31:23.781230 13620 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
I20260806 06:31:23.783645 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 13620
I20260806 06:31:23.783752 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/ts-2/wal/instance
I20260806 06:31:23.783921 7341 external_mini_cluster.cc:1428] Setting key 669cb3374134cd2bc5054598ec7421a7
Aug 06 06:31:23 dist-test-slave-6zbq krb5kdc[13075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997883, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.67@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:31:23.789441 13223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:23.781797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:42333 (local address 127.7.43.126:34557)
0806 06:31:23.781850 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:23.781851 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:23.782025 (+ 174us) server_negotiation.cc:407] Connection header received
0806 06:31:23.782520 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:23.782522 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:23.782534 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:23.782583 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:23.783407 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:23.784114 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:23.785228 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:23.785410 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:23.785460 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:23.787303 (+ 1843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:23.787305 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:23.787305 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:23.787311 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:23.788476 (+ 1165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:23.788799 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:23.788800 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:23.788800 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:23.788843 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:23.789013 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:23.789013 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:23.789013 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:23.789099 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:23.789142 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:23.789328 (+ 186us) server_negotiation.cc:299] Negotiation successful
0806 06:31:23.789381 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 06:31:23.789585 13812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:23.781764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:34557 (local address 127.7.43.67:42333)
0806 06:31:23.781924 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:23.781955 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:23.782433 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:23.782642 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:23.782645 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:23.782878 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:23.783318 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:23.783322 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:23.784215 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:23.784217 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:23.785160 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:23.785163 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:23.785467 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:23.785467 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:23.785551 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:23.785944 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:23.785953 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:23.787217 (+ 1264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:23.788540 (+ 1323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:23.788544 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:23.788546 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:23.788744 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:23.788885 (+ 141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:23.788886 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:23.788886 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:23.788973 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:23.789142 (+ 169us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:23.789145 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:23.789240 (+ 95us) client_negotiation.cc:769] Sending connection context
0806 06:31:23.789311 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 06:31:23.789411 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":90,"threads_started":1}
I20260806 06:31:23.789999 13809 heartbeater.cc:347] Connected to a master server at 127.7.43.126:34557
I20260806 06:31:23.790072 13809 heartbeater.cc:464] Registering TS with master...
I20260806 06:31:23.790199 13809 heartbeater.cc:511] Master 127.7.43.126:34557 requested a full tablet report, sending...
I20260806 06:31:23.790531 13140 ts_manager.cc:194] Registered new tserver with Master: 795001fb9ee24ab38ac90a363195383d (127.7.43.67:42559)
I20260806 06:31:23.790869 13140 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.67@KRBTEST.COM'} at 127.7.43.67:42333
I20260806 06:31:23.795317 7341 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:31:23.801345 13223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:23.796455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50242 (local address 127.7.43.126:34557)
0806 06:31:23.796510 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:23.796512 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:23.796624 (+ 112us) server_negotiation.cc:407] Connection header received
0806 06:31:23.796677 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:23.796677 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:23.796686 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:23.796729 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:23.797228 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:23.797666 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:23.798327 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:23.798543 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:23.798595 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:23.799524 (+ 929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:23.799525 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:23.799525 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:23.799531 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:23.800451 (+ 920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:23.800732 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:23.800733 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:23.800733 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:23.800785 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:23.800966 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:23.800967 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:23.800967 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:23.801082 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:23.801163 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:23.801237 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 06:31:23.801288 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 06:31:23.802683 13140 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50242:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:31:23.804078 13140 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:31:23.809820 13830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:23.805865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35328 (local address 127.7.43.67:42559)
0806 06:31:23.805993 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:23.805997 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:23.806230 (+ 233us) server_negotiation.cc:407] Connection header received
0806 06:31:23.806303 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:23.806305 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:23.806319 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:23.806413 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:31:23.806698 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:23.807731 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:23.809010 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:23.809455 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:23.809518 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:23.809572 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:23.809712 (+ 140us) server_negotiation.cc:299] Negotiation successful
0806 06:31:23.809746 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":76,"threads_started":1}
I20260806 06:31:23.809878 13834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:23.805823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:37049 (local address 127.0.0.1:57572)
0806 06:31:23.806307 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:23.806317 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:23.806395 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:23.806558 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:23.806559 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:23.806560 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:31:23.806589 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:23.806590 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:23.807589 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:23.807590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:23.808798 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:23.808801 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:23.809642 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:23.809644 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:23.809696 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:23.809700 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:31:23.809747 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 06:31:23.809793 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"mutex_wait_us":31,"thread_start_us":59,"threads_started":1}
I20260806 06:31:23.809893 13833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:23.805949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57572 (local address 127.7.43.65:37049)
0806 06:31:23.806121 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:23.806124 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:23.806393 (+ 269us) server_negotiation.cc:407] Connection header received
0806 06:31:23.806463 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:23.806465 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:23.806480 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:23.806556 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:31:23.806637 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:23.807499 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:23.808864 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:23.809502 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:23.809554 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:23.809600 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:23.809774 (+ 174us) server_negotiation.cc:299] Negotiation successful
0806 06:31:23.809814 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":89,"threads_started":1}
I20260806 06:31:23.810204 13828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:23.806002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.66:46449 (local address 127.0.0.1:49452)
0806 06:31:23.806328 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:23.806353 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:23.806397 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:23.806650 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:23.806651 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:23.806653 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:31:23.806699 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:23.806700 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:23.807898 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:23.807899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:23.808665 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:23.808667 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:23.809831 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:23.809832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:23.810041 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:23.810045 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:31:23.810102 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 06:31:23.810140 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":61,"threads_started":1}
I20260806 06:31:23.810230 13835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:23.806134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49452 (local address 127.7.43.66:46449)
0806 06:31:23.806311 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:23.806317 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:23.806446 (+ 129us) server_negotiation.cc:407] Connection header received
0806 06:31:23.806485 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:23.806487 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:23.806503 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:23.806565 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:31:23.806762 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:23.807827 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:23.808746 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:23.809447 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:23.809521 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:23.809581 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:23.810131 (+ 550us) server_negotiation.cc:299] Negotiation successful
0806 06:31:23.810168 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":79,"threads_started":1}
I20260806 06:31:23.810360 13827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:23.805820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.67:42559 (local address 127.0.0.1:35328)
0806 06:31:23.806133 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:23.806164 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:23.806221 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:23.806490 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:23.806493 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:23.806494 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:31:23.806544 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:23.806546 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:23.807823 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:23.807824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:23.808946 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:23.808948 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:23.809541 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:23.809542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:23.809621 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:23.809633 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 06:31:23.809693 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 06:31:23.809754 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"mutex_wait_us":1,"thread_start_us":52,"threads_started":1}
I20260806 06:31:23.810721 13324 tablet_service.cc:1467] Processing CreateTablet for tablet 4f6e93fb3f8f4319b14147bd207af2eb (DEFAULT_TABLE table=test-table [id=f8f1013b09794c57b1229bb11aab5fd6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:31:23.810781 13521 tablet_service.cc:1467] Processing CreateTablet for tablet 4f6e93fb3f8f4319b14147bd207af2eb (DEFAULT_TABLE table=test-table [id=f8f1013b09794c57b1229bb11aab5fd6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:31:23.810896 13714 tablet_service.cc:1467] Processing CreateTablet for tablet 4f6e93fb3f8f4319b14147bd207af2eb (DEFAULT_TABLE table=test-table [id=f8f1013b09794c57b1229bb11aab5fd6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:31:23.811060 13324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f6e93fb3f8f4319b14147bd207af2eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:23.811084 13521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f6e93fb3f8f4319b14147bd207af2eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:23.811154 13714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f6e93fb3f8f4319b14147bd207af2eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:23.813668 13837 tablet_bootstrap.cc:492] T 4f6e93fb3f8f4319b14147bd207af2eb P 148ba296c40b4a9baca9e19364a7d31e: Bootstrap starting.
I20260806 06:31:23.814183 13840 tablet_bootstrap.cc:492] T 4f6e93fb3f8f4319b14147bd207af2eb P 795001fb9ee24ab38ac90a363195383d: Bootstrap starting.
I20260806 06:31:23.814471 13841 tablet_bootstrap.cc:492] T 4f6e93fb3f8f4319b14147bd207af2eb P 4d3751adc69240078920344d64b9058c: Bootstrap starting.
I20260806 06:31:23.814706 13837 tablet_bootstrap.cc:654] T 4f6e93fb3f8f4319b14147bd207af2eb P 148ba296c40b4a9baca9e19364a7d31e: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:23.814757 13840 tablet_bootstrap.cc:654] T 4f6e93fb3f8f4319b14147bd207af2eb P 795001fb9ee24ab38ac90a363195383d: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:23.815001 13841 tablet_bootstrap.cc:654] T 4f6e93fb3f8f4319b14147bd207af2eb P 4d3751adc69240078920344d64b9058c: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:23.815330 13837 log.cc:824] T 4f6e93fb3f8f4319b14147bd207af2eb P 148ba296c40b4a9baca9e19364a7d31e: Log is configured to *not* fsync() on all Append() calls
I20260806 06:31:23.815492 13841 log.cc:824] T 4f6e93fb3f8f4319b14147bd207af2eb P 4d3751adc69240078920344d64b9058c: Log is configured to *not* fsync() on all Append() calls
I20260806 06:31:23.816020 13837 tablet_bootstrap.cc:492] T 4f6e93fb3f8f4319b14147bd207af2eb P 148ba296c40b4a9baca9e19364a7d31e: No bootstrap required, opened a new log
I20260806 06:31:23.816095 13837 ts_tablet_manager.cc:1397] T 4f6e93fb3f8f4319b14147bd207af2eb P 148ba296c40b4a9baca9e19364a7d31e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 06:31:23.816161 13841 tablet_bootstrap.cc:492] T 4f6e93fb3f8f4319b14147bd207af2eb P 4d3751adc69240078920344d64b9058c: No bootstrap required, opened a new log
I20260806 06:31:23.816223 13841 ts_tablet_manager.cc:1397] T 4f6e93fb3f8f4319b14147bd207af2eb P 4d3751adc69240078920344d64b9058c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:31:23.817425 13840 log.cc:824] T 4f6e93fb3f8f4319b14147bd207af2eb P 795001fb9ee24ab38ac90a363195383d: Log is configured to *not* fsync() on all Append() calls
I20260806 06:31:23.818063 13840 tablet_bootstrap.cc:492] T 4f6e93fb3f8f4319b14147bd207af2eb P 795001fb9ee24ab38ac90a363195383d: No bootstrap required, opened a new log
I20260806 06:31:23.818130 13840 ts_tablet_manager.cc:1397] T 4f6e93fb3f8f4319b14147bd207af2eb P 795001fb9ee24ab38ac90a363195383d: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20260806 06:31:23.818017 13837 raft_consensus.cc:348] T 4f6e93fb3f8f4319b14147bd207af2eb P 148ba296c40b4a9baca9e19364a7d31e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d3751adc69240078920344d64b9058c" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 37049 } } peers { permanent_uuid: "795001fb9ee24ab38ac90a363195383d" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 42559 } } peers { permanent_uuid: "148ba296c40b4a9baca9e19364a7d31e" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 46449 } }
I20260806 06:31:23.818055 13841 raft_consensus.cc:348] T 4f6e93fb3f8f4319b14147bd207af2eb P 4d3751adc69240078920344d64b9058c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d3751adc69240078920344d64b9058c" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 37049 } } peers { permanent_uuid: "795001fb9ee24ab38ac90a363195383d" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 42559 } } peers { permanent_uuid: "148ba296c40b4a9baca9e19364a7d31e" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 46449 } }
I20260806 06:31:23.818176 13841 raft_consensus.cc:374] T 4f6e93fb3f8f4319b14147bd207af2eb P 4d3751adc69240078920344d64b9058c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:23.818176 13837 raft_consensus.cc:374] T 4f6e93fb3f8f4319b14147bd207af2eb P 148ba296c40b4a9baca9e19364a7d31e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:23.818217 13841 raft_consensus.cc:729] T 4f6e93fb3f8f4319b14147bd207af2eb P 4d3751adc69240078920344d64b9058c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d3751adc69240078920344d64b9058c, State: Initialized, Role: FOLLOWER
I20260806 06:31:23.818217 13837 raft_consensus.cc:729] T 4f6e93fb3f8f4319b14147bd207af2eb P 148ba296c40b4a9baca9e19364a7d31e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 148ba296c40b4a9baca9e19364a7d31e, State: Initialized, Role: FOLLOWER
I20260806 06:31:23.818372 13841 consensus_queue.cc:260] T 4f6e93fb3f8f4319b14147bd207af2eb P 4d3751adc69240078920344d64b9058c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d3751adc69240078920344d64b9058c" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 37049 } } peers { permanent_uuid: "795001fb9ee24ab38ac90a363195383d" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 42559 } } peers { permanent_uuid: "148ba296c40b4a9baca9e19364a7d31e" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 46449 } }
I20260806 06:31:23.818368 13837 consensus_queue.cc:260] T 4f6e93fb3f8f4319b14147bd207af2eb P 148ba296c40b4a9baca9e19364a7d31e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d3751adc69240078920344d64b9058c" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 37049 } } peers { permanent_uuid: "795001fb9ee24ab38ac90a363195383d" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 42559 } } peers { permanent_uuid: "148ba296c40b4a9baca9e19364a7d31e" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 46449 } }
I20260806 06:31:23.818652 13841 ts_tablet_manager.cc:1428] T 4f6e93fb3f8f4319b14147bd207af2eb P 4d3751adc69240078920344d64b9058c: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 06:31:23.818652 13837 ts_tablet_manager.cc:1428] T 4f6e93fb3f8f4319b14147bd207af2eb P 148ba296c40b4a9baca9e19364a7d31e: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260806 06:31:23.818771 13421 heartbeater.cc:503] Master 127.7.43.126:34557 was elected leader, sending a full tablet report...
I20260806 06:31:23.819131 13613 heartbeater.cc:503] Master 127.7.43.126:34557 was elected leader, sending a full tablet report...
I20260806 06:31:23.819983 13840 raft_consensus.cc:348] T 4f6e93fb3f8f4319b14147bd207af2eb P 795001fb9ee24ab38ac90a363195383d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d3751adc69240078920344d64b9058c" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 37049 } } peers { permanent_uuid: "795001fb9ee24ab38ac90a363195383d" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 42559 } } peers { permanent_uuid: "148ba296c40b4a9baca9e19364a7d31e" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 46449 } }
I20260806 06:31:23.820103 13840 raft_consensus.cc:374] T 4f6e93fb3f8f4319b14147bd207af2eb P 795001fb9ee24ab38ac90a363195383d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:23.820144 13840 raft_consensus.cc:729] T 4f6e93fb3f8f4319b14147bd207af2eb P 795001fb9ee24ab38ac90a363195383d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 795001fb9ee24ab38ac90a363195383d, State: Initialized, Role: FOLLOWER
I20260806 06:31:23.820253 13840 consensus_queue.cc:260] T 4f6e93fb3f8f4319b14147bd207af2eb P 795001fb9ee24ab38ac90a363195383d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d3751adc69240078920344d64b9058c" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 37049 } } peers { permanent_uuid: "795001fb9ee24ab38ac90a363195383d" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 42559 } } peers { permanent_uuid: "148ba296c40b4a9baca9e19364a7d31e" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 46449 } }
I20260806 06:31:23.820475 13840 ts_tablet_manager.cc:1428] T 4f6e93fb3f8f4319b14147bd207af2eb P 795001fb9ee24ab38ac90a363195383d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 06:31:23.820501 13809 heartbeater.cc:503] Master 127.7.43.126:34557 was elected leader, sending a full tablet report...
W20260806 06:31:23.895162 13614 tablet.cc:2369] T 4f6e93fb3f8f4319b14147bd207af2eb P 148ba296c40b4a9baca9e19364a7d31e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 06:31:23.991883 13422 tablet.cc:2369] T 4f6e93fb3f8f4319b14147bd207af2eb P 4d3751adc69240078920344d64b9058c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:31:24.006830 13847 raft_consensus.cc:482] T 4f6e93fb3f8f4319b14147bd207af2eb P 795001fb9ee24ab38ac90a363195383d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:31:24.006942 13847 raft_consensus.cc:504] T 4f6e93fb3f8f4319b14147bd207af2eb P 795001fb9ee24ab38ac90a363195383d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d3751adc69240078920344d64b9058c" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 37049 } } peers { permanent_uuid: "795001fb9ee24ab38ac90a363195383d" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 42559 } } peers { permanent_uuid: "148ba296c40b4a9baca9e19364a7d31e" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 46449 } }
I20260806 06:31:24.007287 13847 leader_election.cc:288] T 4f6e93fb3f8f4319b14147bd207af2eb P 795001fb9ee24ab38ac90a363195383d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d3751adc69240078920344d64b9058c (127.7.43.65:37049), 148ba296c40b4a9baca9e19364a7d31e (127.7.43.66:46449)
I20260806 06:31:24.010908 13833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:24.007473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:49339 (local address 127.7.43.65:37049)
0806 06:31:24.007529 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:24.007532 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:24.007576 (+ 44us) server_negotiation.cc:407] Connection header received
0806 06:31:24.007653 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:24.007654 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:24.007668 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:24.007722 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:31:24.007880 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:24.008627 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:24.009995 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:24.010575 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:24.010647 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:24.010657 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:24.010804 (+ 147us) server_negotiation.cc:299] Negotiation successful
0806 06:31:24.010840 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 06:31:24.010911 13835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:24.007504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:57289 (local address 127.7.43.66:46449)
0806 06:31:24.007554 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:24.007556 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:24.007758 (+ 202us) server_negotiation.cc:407] Connection header received
0806 06:31:24.007805 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:24.007806 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:24.007817 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:24.007874 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:31:24.007971 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:24.008657 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:24.009995 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:24.010575 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:24.010646 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:24.010657 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:24.010804 (+ 147us) server_negotiation.cc:299] Negotiation successful
0806 06:31:24.010853 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 06:31:24.010946 13812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:24.007450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:37049 (local address 127.7.43.67:49339)
0806 06:31:24.007523 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:24.007535 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:24.007586 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:24.007740 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:24.007741 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:24.007743 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:31:24.007815 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:24.007818 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:24.008700 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:24.008701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:24.009914 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:24.009917 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:24.010644 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:24.010645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:24.010738 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:24.010740 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:31:24.010793 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 06:31:24.010844 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55,"mutex_wait_us":6}
I20260806 06:31:24.011114 13851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:24.007458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.66:46449 (local address 127.7.43.67:57289)
0806 06:31:24.007690 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:24.007698 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:24.007758 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:24.007882 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:24.007883 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:24.007884 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:31:24.007930 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:24.007931 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:24.008712 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:24.008712 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:24.009914 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:24.009917 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:24.010645 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:24.010645 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:24.010725 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:24.010729 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:31:24.010793 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 06:31:24.010844 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"mutex_wait_us":113,"thread_start_us":98,"threads_started":1}
I20260806 06:31:24.011277 13350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f6e93fb3f8f4319b14147bd207af2eb" candidate_uuid: "795001fb9ee24ab38ac90a363195383d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d3751adc69240078920344d64b9058c" is_pre_election: true
I20260806 06:31:24.011430 13350 raft_consensus.cc:2454] T 4f6e93fb3f8f4319b14147bd207af2eb P 4d3751adc69240078920344d64b9058c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 795001fb9ee24ab38ac90a363195383d in term 0.
I20260806 06:31:24.011476 13544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f6e93fb3f8f4319b14147bd207af2eb" candidate_uuid: "795001fb9ee24ab38ac90a363195383d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "148ba296c40b4a9baca9e19364a7d31e" is_pre_election: true
I20260806 06:31:24.011615 13544 raft_consensus.cc:2454] T 4f6e93fb3f8f4319b14147bd207af2eb P 148ba296c40b4a9baca9e19364a7d31e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 795001fb9ee24ab38ac90a363195383d in term 0.
I20260806 06:31:24.011662 13641 leader_election.cc:302] T 4f6e93fb3f8f4319b14147bd207af2eb P 795001fb9ee24ab38ac90a363195383d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d3751adc69240078920344d64b9058c, 795001fb9ee24ab38ac90a363195383d; no voters:
I20260806 06:31:24.011798 13847 raft_consensus.cc:2781] T 4f6e93fb3f8f4319b14147bd207af2eb P 795001fb9ee24ab38ac90a363195383d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:31:24.011863 13847 raft_consensus.cc:482] T 4f6e93fb3f8f4319b14147bd207af2eb P 795001fb9ee24ab38ac90a363195383d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:31:24.011885 13847 raft_consensus.cc:3037] T 4f6e93fb3f8f4319b14147bd207af2eb P 795001fb9ee24ab38ac90a363195383d [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:24.012636 13847 raft_consensus.cc:504] T 4f6e93fb3f8f4319b14147bd207af2eb P 795001fb9ee24ab38ac90a363195383d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d3751adc69240078920344d64b9058c" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 37049 } } peers { permanent_uuid: "795001fb9ee24ab38ac90a363195383d" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 42559 } } peers { permanent_uuid: "148ba296c40b4a9baca9e19364a7d31e" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 46449 } }
I20260806 06:31:24.012756 13847 leader_election.cc:288] T 4f6e93fb3f8f4319b14147bd207af2eb P 795001fb9ee24ab38ac90a363195383d [CANDIDATE]: Term 1 election: Requested vote from peers 4d3751adc69240078920344d64b9058c (127.7.43.65:37049), 148ba296c40b4a9baca9e19364a7d31e (127.7.43.66:46449)
I20260806 06:31:24.012900 13350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f6e93fb3f8f4319b14147bd207af2eb" candidate_uuid: "795001fb9ee24ab38ac90a363195383d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d3751adc69240078920344d64b9058c"
I20260806 06:31:24.012900 13544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f6e93fb3f8f4319b14147bd207af2eb" candidate_uuid: "795001fb9ee24ab38ac90a363195383d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "148ba296c40b4a9baca9e19364a7d31e"
I20260806 06:31:24.012984 13350 raft_consensus.cc:3037] T 4f6e93fb3f8f4319b14147bd207af2eb P 4d3751adc69240078920344d64b9058c [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:24.012996 13544 raft_consensus.cc:3037] T 4f6e93fb3f8f4319b14147bd207af2eb P 148ba296c40b4a9baca9e19364a7d31e [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:24.013846 13544 raft_consensus.cc:2454] T 4f6e93fb3f8f4319b14147bd207af2eb P 148ba296c40b4a9baca9e19364a7d31e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 795001fb9ee24ab38ac90a363195383d in term 1.
I20260806 06:31:24.013851 13350 raft_consensus.cc:2454] T 4f6e93fb3f8f4319b14147bd207af2eb P 4d3751adc69240078920344d64b9058c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 795001fb9ee24ab38ac90a363195383d in term 1.
I20260806 06:31:24.014007 13640 leader_election.cc:302] T 4f6e93fb3f8f4319b14147bd207af2eb P 795001fb9ee24ab38ac90a363195383d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 148ba296c40b4a9baca9e19364a7d31e, 795001fb9ee24ab38ac90a363195383d; no voters:
I20260806 06:31:24.014122 13847 raft_consensus.cc:2781] T 4f6e93fb3f8f4319b14147bd207af2eb P 795001fb9ee24ab38ac90a363195383d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:31:24.014248 13847 raft_consensus.cc:686] T 4f6e93fb3f8f4319b14147bd207af2eb P 795001fb9ee24ab38ac90a363195383d [term 1 LEADER]: Becoming Leader. State: Replica: 795001fb9ee24ab38ac90a363195383d, State: Running, Role: LEADER
I20260806 06:31:24.014330 13847 consensus_queue.cc:237] T 4f6e93fb3f8f4319b14147bd207af2eb P 795001fb9ee24ab38ac90a363195383d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d3751adc69240078920344d64b9058c" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 37049 } } peers { permanent_uuid: "795001fb9ee24ab38ac90a363195383d" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 42559 } } peers { permanent_uuid: "148ba296c40b4a9baca9e19364a7d31e" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 46449 } }
I20260806 06:31:24.015115 13140 catalog_manager.cc:5495] T 4f6e93fb3f8f4319b14147bd207af2eb P 795001fb9ee24ab38ac90a363195383d reported cstate change: term changed from 0 to 1, leader changed from <none> to 795001fb9ee24ab38ac90a363195383d (127.7.43.67). New cstate: current_term: 1 leader_uuid: "795001fb9ee24ab38ac90a363195383d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d3751adc69240078920344d64b9058c" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 37049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "795001fb9ee24ab38ac90a363195383d" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 42559 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "148ba296c40b4a9baca9e19364a7d31e" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 46449 } health_report { overall_health: UNKNOWN } } }
W20260806 06:31:24.031693 13810 tablet.cc:2369] T 4f6e93fb3f8f4319b14147bd207af2eb P 795001fb9ee24ab38ac90a363195383d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:31:24.032418 13830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:24.030222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35336 (local address 127.7.43.67:42559)
0806 06:31:24.030267 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:24.030269 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:24.030275 (+ 6us) server_negotiation.cc:407] Connection header received
0806 06:31:24.030402 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:24.030403 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:24.030423 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:24.030495 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:31:24.030599 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:24.031027 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:24.031781 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:24.031999 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:24.032052 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:24.032184 (+ 132us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:31:24.032270 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:31:24.032324 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:24.032341 (+ 17us) server_negotiation.cc:299] Negotiation successful
0806 06:31:24.032356 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 06:31:24.033883 13544 raft_consensus.cc:1261] T 4f6e93fb3f8f4319b14147bd207af2eb P 148ba296c40b4a9baca9e19364a7d31e [term 1 FOLLOWER]: Refusing update from remote peer 795001fb9ee24ab38ac90a363195383d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:31:24.033885 13350 raft_consensus.cc:1261] T 4f6e93fb3f8f4319b14147bd207af2eb P 4d3751adc69240078920344d64b9058c [term 1 FOLLOWER]: Refusing update from remote peer 795001fb9ee24ab38ac90a363195383d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:31:24.034161 13847 consensus_queue.cc:1035] T 4f6e93fb3f8f4319b14147bd207af2eb P 795001fb9ee24ab38ac90a363195383d [LEADER]: Connected to new peer: Peer: permanent_uuid: "148ba296c40b4a9baca9e19364a7d31e" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 46449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:31:24.034278 13852 consensus_queue.cc:1035] T 4f6e93fb3f8f4319b14147bd207af2eb P 795001fb9ee24ab38ac90a363195383d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d3751adc69240078920344d64b9058c" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 37049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:31:24.036574 13856 mvcc.cc:205] Tried to move back new op lower bound from 7315447333000630272 to 7315447332923330560. Current Snapshot: MvccSnapshot[applied={T|T < 7315447333000630272}]
I20260806 06:31:24.036763 13859 mvcc.cc:205] Tried to move back new op lower bound from 7315447333000630272 to 7315447332923330560. Current Snapshot: MvccSnapshot[applied={T|T < 7315447333000630272}]
I20260806 06:31:24.038403 13138 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50242:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:31:24.038496 13138 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50242:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:31:24.039222 13138 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d2ecafe8d84c4c05ba95ea992b1b8f26: Sending DeleteTablet for 3 replicas of tablet 4f6e93fb3f8f4319b14147bd207af2eb
I20260806 06:31:24.039516 13714 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f6e93fb3f8f4319b14147bd207af2eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:31:24 UTC) from {username='kudu', principal='kudu/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:35328
I20260806 06:31:24.039547 13324 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f6e93fb3f8f4319b14147bd207af2eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:31:24 UTC) from {username='kudu', principal='kudu/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:57572
I20260806 06:31:24.039552 13521 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f6e93fb3f8f4319b14147bd207af2eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:31:24 UTC) from {username='kudu', principal='kudu/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:49452
I20260806 06:31:24.039762 13872 tablet_replica.cc:330] T 4f6e93fb3f8f4319b14147bd207af2eb P 148ba296c40b4a9baca9e19364a7d31e: stopping tablet replica
I20260806 06:31:24.039827 13871 tablet_replica.cc:330] T 4f6e93fb3f8f4319b14147bd207af2eb P 795001fb9ee24ab38ac90a363195383d: stopping tablet replica
I20260806 06:31:24.039861 13872 raft_consensus.cc:2229] T 4f6e93fb3f8f4319b14147bd207af2eb P 148ba296c40b4a9baca9e19364a7d31e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:31:24.039918 13871 raft_consensus.cc:2229] T 4f6e93fb3f8f4319b14147bd207af2eb P 795001fb9ee24ab38ac90a363195383d [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:31:24.039950 13872 raft_consensus.cc:2258] T 4f6e93fb3f8f4319b14147bd207af2eb P 148ba296c40b4a9baca9e19364a7d31e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:31:24.040022 13871 raft_consensus.cc:2258] T 4f6e93fb3f8f4319b14147bd207af2eb P 795001fb9ee24ab38ac90a363195383d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:31:24.040190 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 13235
I20260806 06:31:24.040381 13872 ts_tablet_manager.cc:1905] T 4f6e93fb3f8f4319b14147bd207af2eb P 148ba296c40b4a9baca9e19364a7d31e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:31:24.040439 13871 ts_tablet_manager.cc:1905] T 4f6e93fb3f8f4319b14147bd207af2eb P 795001fb9ee24ab38ac90a363195383d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:31:24.041910 13872 ts_tablet_manager.cc:1918] T 4f6e93fb3f8f4319b14147bd207af2eb P 148ba296c40b4a9baca9e19364a7d31e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:31:24.041970 13872 log.cc:1196] T 4f6e93fb3f8f4319b14147bd207af2eb P 148ba296c40b4a9baca9e19364a7d31e: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/ts-1/wal/wals/4f6e93fb3f8f4319b14147bd207af2eb
I20260806 06:31:24.042191 13871 ts_tablet_manager.cc:1918] T 4f6e93fb3f8f4319b14147bd207af2eb P 795001fb9ee24ab38ac90a363195383d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:31:24.042222 13872 ts_tablet_manager.cc:1939] T 4f6e93fb3f8f4319b14147bd207af2eb P 148ba296c40b4a9baca9e19364a7d31e: Deleting consensus metadata
I20260806 06:31:24.042263 13871 log.cc:1196] T 4f6e93fb3f8f4319b14147bd207af2eb P 795001fb9ee24ab38ac90a363195383d: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785997861828271-7341-0/minicluster-data/ts-2/wal/wals/4f6e93fb3f8f4319b14147bd207af2eb
I20260806 06:31:24.042640 13118 catalog_manager.cc:4841] TS 148ba296c40b4a9baca9e19364a7d31e (127.7.43.66:46449): tablet 4f6e93fb3f8f4319b14147bd207af2eb (table test-table [id=f8f1013b09794c57b1229bb11aab5fd6]) successfully deleted
I20260806 06:31:24.042680 13871 ts_tablet_manager.cc:1939] T 4f6e93fb3f8f4319b14147bd207af2eb P 795001fb9ee24ab38ac90a363195383d: Deleting consensus metadata
I20260806 06:31:24.043093 13118 catalog_manager.cc:4841] TS 795001fb9ee24ab38ac90a363195383d (127.7.43.67:42559): tablet 4f6e93fb3f8f4319b14147bd207af2eb (table test-table [id=f8f1013b09794c57b1229bb11aab5fd6]) successfully deleted
W20260806 06:31:24.053690 13119 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.43.65:37049 (error 108)
I20260806 06:31:24.053828 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 13427
I20260806 06:31:24.057718 13827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:24.057537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:37049 (local address 127.0.0.1:57586)
0806 06:31:24.057611 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:24.057679 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.65:37049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260806 06:31:24.057868 13119 catalog_manager.cc:4584] TS 4d3751adc69240078920344d64b9058c (127.7.43.65:37049): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4f6e93fb3f8f4319b14147bd207af2eb: Network error: Client connection negotiation failed: client connection to 127.7.43.65:37049: connect: Connection refused (error 111)
I20260806 06:31:24.062479 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 13620
I20260806 06:31:24.070276 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 13098
2026-08-06T06:31:24Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2094 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-06T06:31:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:31:24Z Disabled control of system clock
I20260806 06:31:24.100054 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:43045
--webserver_interface=127.7.43.126
--webserver_port=0
--builtin_ntp_servers=127.7.43.84:38139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.43.126:43045
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 06:31:24.178398 13882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:24.178580 13882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:24.178599 13882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:24.179658 13882 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:31:24.179693 13882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:24.179704 13882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:24.179716 13882 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:31:24.179728 13882 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:31:24.180985 13882 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:38139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.43.126:43045
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:43045
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.43.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13882
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:24.181186 13882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:24.181417 13882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:24.185698 13894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:24.185715 13889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:24.185782 13890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:24.185995 13882 server_base.cc:1034] running on GCE node
I20260806 06:31:24.186205 13882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:24.186502 13882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:24.187636 13882 hybrid_clock.cc:648] HybridClock initialized: now 1785997884187621 us; error 31 us; skew 500 ppm
I20260806 06:31:24.188721 13882 webserver.cc:460] Webserver started at http://127.7.43.126:43041/ using document root <none> and password file <none>
I20260806 06:31:24.188951 13882 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:24.188999 13882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:24.189119 13882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:24.190089 13882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/master-0/data/instance:
uuid: "fade6d4ef3bd4bc790b118e0dc805a3a"
format_stamp: "Formatted at 2026-08-06 06:31:24 on dist-test-slave-6zbq"
server_key: "29cde1d8f7da56a62d3b274b5caa3f8a"
server_key_iv: "443e5deeff45f343989bd593d3ac2b80"
server_key_version: "encryptionkey@0"
I20260806 06:31:24.190442 13882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/master-0/wal/instance:
uuid: "fade6d4ef3bd4bc790b118e0dc805a3a"
format_stamp: "Formatted at 2026-08-06 06:31:24 on dist-test-slave-6zbq"
server_key: "29cde1d8f7da56a62d3b274b5caa3f8a"
server_key_iv: "443e5deeff45f343989bd593d3ac2b80"
server_key_version: "encryptionkey@0"
I20260806 06:31:24.191583 13882 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:31:24.192221 13902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:24.192356 13882 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:31:24.192440 13882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "fade6d4ef3bd4bc790b118e0dc805a3a"
format_stamp: "Formatted at 2026-08-06 06:31:24 on dist-test-slave-6zbq"
server_key: "29cde1d8f7da56a62d3b274b5caa3f8a"
server_key_iv: "443e5deeff45f343989bd593d3ac2b80"
server_key_version: "encryptionkey@0"
I20260806 06:31:24.192507 13882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:24.195698 13882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:24.195919 13882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:24.196033 13882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:24.199519 13882 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.126:43045
I20260806 06:31:24.199550 13971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.126:43045 every 8 connection(s)
I20260806 06:31:24.199827 13882 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:31:24.200412 13972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:24.203058 13972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fade6d4ef3bd4bc790b118e0dc805a3a: Bootstrap starting.
I20260806 06:31:24.203572 13972 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fade6d4ef3bd4bc790b118e0dc805a3a: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:24.203791 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 13882
I20260806 06:31:24.203851 13972 log.cc:824] T 00000000000000000000000000000000 P fade6d4ef3bd4bc790b118e0dc805a3a: Log is configured to *not* fsync() on all Append() calls
I20260806 06:31:24.203881 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/master-0/wal/instance
I20260806 06:31:24.204044 7341 external_mini_cluster.cc:1428] Setting key 03e7cbf2ddf07c8c07110d61768015a0
I20260806 06:31:24.204548 13972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fade6d4ef3bd4bc790b118e0dc805a3a: No bootstrap required, opened a new log
I20260806 06:31:24.206491 13972 raft_consensus.cc:348] T 00000000000000000000000000000000 P fade6d4ef3bd4bc790b118e0dc805a3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fade6d4ef3bd4bc790b118e0dc805a3a" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 43045 } }
I20260806 06:31:24.206617 13972 raft_consensus.cc:374] T 00000000000000000000000000000000 P fade6d4ef3bd4bc790b118e0dc805a3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:24.206650 13972 raft_consensus.cc:729] T 00000000000000000000000000000000 P fade6d4ef3bd4bc790b118e0dc805a3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fade6d4ef3bd4bc790b118e0dc805a3a, State: Initialized, Role: FOLLOWER
I20260806 06:31:24.206789 13972 consensus_queue.cc:260] T 00000000000000000000000000000000 P fade6d4ef3bd4bc790b118e0dc805a3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fade6d4ef3bd4bc790b118e0dc805a3a" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 43045 } }
I20260806 06:31:24.206897 13972 raft_consensus.cc:388] T 00000000000000000000000000000000 P fade6d4ef3bd4bc790b118e0dc805a3a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:31:24.206911 13977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:24.204470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54860 (local address 127.7.43.126:43045)
0806 06:31:24.204677 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:24.204685 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:24.204711 (+ 26us) server_negotiation.cc:407] Connection header received
0806 06:31:24.205665 (+ 954us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:24.205673 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:24.205710 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:24.205826 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:24.206484 (+ 658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:24.206488 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:24.206488 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:31:24.206492 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:24.206535 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:31:24.206544 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:24.206600 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:24.206670 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 06:31:24.206729 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":76,"threads_started":1}
I20260806 06:31:24.206930 13972 raft_consensus.cc:482] T 00000000000000000000000000000000 P fade6d4ef3bd4bc790b118e0dc805a3a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:31:24.207001 13972 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fade6d4ef3bd4bc790b118e0dc805a3a [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:24.207629 13972 raft_consensus.cc:504] T 00000000000000000000000000000000 P fade6d4ef3bd4bc790b118e0dc805a3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fade6d4ef3bd4bc790b118e0dc805a3a" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 43045 } }
I20260806 06:31:24.207738 13972 leader_election.cc:302] T 00000000000000000000000000000000 P fade6d4ef3bd4bc790b118e0dc805a3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fade6d4ef3bd4bc790b118e0dc805a3a; no voters:
I20260806 06:31:24.207904 13972 leader_election.cc:288] T 00000000000000000000000000000000 P fade6d4ef3bd4bc790b118e0dc805a3a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:31:24.207929 13981 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fade6d4ef3bd4bc790b118e0dc805a3a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:31:24.208039 13981 raft_consensus.cc:686] T 00000000000000000000000000000000 P fade6d4ef3bd4bc790b118e0dc805a3a [term 1 LEADER]: Becoming Leader. State: Replica: fade6d4ef3bd4bc790b118e0dc805a3a, State: Running, Role: LEADER
I20260806 06:31:24.208150 13981 consensus_queue.cc:237] T 00000000000000000000000000000000 P fade6d4ef3bd4bc790b118e0dc805a3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fade6d4ef3bd4bc790b118e0dc805a3a" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 43045 } }
I20260806 06:31:24.208184 13972 sys_catalog.cc:564] T 00000000000000000000000000000000 P fade6d4ef3bd4bc790b118e0dc805a3a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:31:24.208521 13982 sys_catalog.cc:455] T 00000000000000000000000000000000 P fade6d4ef3bd4bc790b118e0dc805a3a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fade6d4ef3bd4bc790b118e0dc805a3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fade6d4ef3bd4bc790b118e0dc805a3a" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 43045 } } }
I20260806 06:31:24.208535 13983 sys_catalog.cc:455] T 00000000000000000000000000000000 P fade6d4ef3bd4bc790b118e0dc805a3a [sys.catalog]: SysCatalogTable state changed. Reason: New leader fade6d4ef3bd4bc790b118e0dc805a3a. Latest consensus state: current_term: 1 leader_uuid: "fade6d4ef3bd4bc790b118e0dc805a3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fade6d4ef3bd4bc790b118e0dc805a3a" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 43045 } } }
I20260806 06:31:24.208681 13983 sys_catalog.cc:458] T 00000000000000000000000000000000 P fade6d4ef3bd4bc790b118e0dc805a3a [sys.catalog]: This master's current role is: LEADER
I20260806 06:31:24.208892 13991 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:31:24.209043 13982 sys_catalog.cc:458] T 00000000000000000000000000000000 P fade6d4ef3bd4bc790b118e0dc805a3a [sys.catalog]: This master's current role is: LEADER
I20260806 06:31:24.209461 13991 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:31:24.210795 13991 catalog_manager.cc:1347] Generated new cluster ID: 00e67ee5ade44081b1bd8aa8a2297999
I20260806 06:31:24.210839 13991 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:31:24.216924 13991 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:31:24.217053 13991 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:31:24.223804 13991 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fade6d4ef3bd4bc790b118e0dc805a3a: Generated new TSK 0
I20260806 06:31:24.223950 13991 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:31:24.228423 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.65:0
--local_ip_for_outbound_sockets=127.7.43.65
--webserver_interface=127.7.43.65
--webserver_port=0
--tserver_master_addrs=127.7.43.126:43045
--builtin_ntp_servers=127.7.43.84:38139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 06:31:24.329155 14010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:24.329345 14010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:24.329375 14010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:24.330525 14010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:24.330564 14010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:24.330598 14010 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.65
I20260806 06:31:24.331924 14010 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:38139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.43.65
--webserver_port=0
--tserver_master_addrs=127.7.43.126:43045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14010
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.65
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:24.332139 14010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:24.332391 14010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:24.337170 14021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:24.337342 14010 server_base.cc:1034] running on GCE node
W20260806 06:31:24.337180 14024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:24.337209 14020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:24.337596 14010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:24.337827 14010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:24.338970 14010 hybrid_clock.cc:648] HybridClock initialized: now 1785997884338959 us; error 32 us; skew 500 ppm
I20260806 06:31:24.340096 14010 webserver.cc:460] Webserver started at http://127.7.43.65:34387/ using document root <none> and password file <none>
I20260806 06:31:24.340358 14010 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:24.340415 14010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:24.340551 14010 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:24.341588 14010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/ts-0/data/instance:
uuid: "eeb602dd23a94eb5a8d4b3e76b784c0d"
format_stamp: "Formatted at 2026-08-06 06:31:24 on dist-test-slave-6zbq"
server_key: "26c597d8243ef98979c9fc2482e3b699"
server_key_iv: "38c754add951a89e4cdefa1ac165f7d8"
server_key_version: "encryptionkey@0"
I20260806 06:31:24.341913 14010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/ts-0/wal/instance:
uuid: "eeb602dd23a94eb5a8d4b3e76b784c0d"
format_stamp: "Formatted at 2026-08-06 06:31:24 on dist-test-slave-6zbq"
server_key: "26c597d8243ef98979c9fc2482e3b699"
server_key_iv: "38c754add951a89e4cdefa1ac165f7d8"
server_key_version: "encryptionkey@0"
I20260806 06:31:24.343168 14010 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:31:24.343869 14034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:24.344058 14010 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:31:24.344142 14010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/ts-0/wal
uuid: "eeb602dd23a94eb5a8d4b3e76b784c0d"
format_stamp: "Formatted at 2026-08-06 06:31:24 on dist-test-slave-6zbq"
server_key: "26c597d8243ef98979c9fc2482e3b699"
server_key_iv: "38c754add951a89e4cdefa1ac165f7d8"
server_key_version: "encryptionkey@0"
I20260806 06:31:24.344208 14010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:24.347404 14010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:24.347638 14010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:24.347756 14010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:24.347957 14010 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:31:24.348239 14010 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:31:24.348272 14010 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:24.348299 14010 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:31:24.348323 14010 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:24.354306 14010 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.65:34161
I20260806 06:31:24.354372 14207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.65:34161 every 8 connection(s)
I20260806 06:31:24.354648 14010 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
I20260806 06:31:24.357262 13977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:24.355265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:38753 (local address 127.7.43.126:43045)
0806 06:31:24.355325 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:24.355326 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:24.355541 (+ 215us) server_negotiation.cc:407] Connection header received
0806 06:31:24.355974 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:24.355976 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:24.355982 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:24.356032 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:24.357029 (+ 997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:24.357031 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:24.357031 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:31:24.357035 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:24.357051 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:31:24.357055 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:24.357104 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:24.357181 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 06:31:24.357194 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 06:31:24.357440 14210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:24.355251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:43045 (local address 127.7.43.65:38753)
0806 06:31:24.355431 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:24.355460 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:24.355882 (+ 422us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:24.356095 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:24.356098 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:24.356106 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:24.356890 (+ 784us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:31:24.356890 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:24.356916 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:31:24.356942 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:24.357108 (+ 166us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:24.357113 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:24.357116 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:31:24.357184 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 06:31:24.357250 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":89,"threads_started":1}
I20260806 06:31:24.357882 14208 heartbeater.cc:347] Connected to a master server at 127.7.43.126:43045
I20260806 06:31:24.357954 14208 heartbeater.cc:464] Registering TS with master...
I20260806 06:31:24.358088 14208 heartbeater.cc:511] Master 127.7.43.126:43045 requested a full tablet report, sending...
I20260806 06:31:24.358566 13927 ts_manager.cc:194] Registered new tserver with Master: eeb602dd23a94eb5a8d4b3e76b784c0d (127.7.43.65:34161)
I20260806 06:31:24.364146 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 14010
I20260806 06:31:24.364228 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/ts-0/wal/instance
I20260806 06:31:24.364367 7341 external_mini_cluster.cc:1428] Setting key 0cefbdf20e14d3a353e3d60ea8c99cb3
I20260806 06:31:24.365073 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.66:0
--local_ip_for_outbound_sockets=127.7.43.66
--webserver_interface=127.7.43.66
--webserver_port=0
--tserver_master_addrs=127.7.43.126:43045
--builtin_ntp_servers=127.7.43.84:38139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 06:31:24.444502 14211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:24.444679 14211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:24.444700 14211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:24.445749 14211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:24.445780 14211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:24.445804 14211 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.66
I20260806 06:31:24.447222 14211 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:38139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.43.66
--webserver_port=0
--tserver_master_addrs=127.7.43.126:43045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14211
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.66
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:24.447402 14211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:24.447625 14211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:24.452167 14223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:24.452173 14221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:24.452183 14220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:24.452394 14211 server_base.cc:1034] running on GCE node
I20260806 06:31:24.452591 14211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:24.452812 14211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:24.453955 14211 hybrid_clock.cc:648] HybridClock initialized: now 1785997884453936 us; error 33 us; skew 500 ppm
I20260806 06:31:24.455127 14211 webserver.cc:460] Webserver started at http://127.7.43.66:36461/ using document root <none> and password file <none>
I20260806 06:31:24.455343 14211 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:24.455387 14211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:24.455498 14211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:24.456467 14211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/ts-1/data/instance:
uuid: "98c66aa965b54ff79c1ff9677e6e94ba"
format_stamp: "Formatted at 2026-08-06 06:31:24 on dist-test-slave-6zbq"
server_key: "2860bd804e1fa5cda4beb8073a3f749f"
server_key_iv: "f276b8e8b4d02193b3ed84fa2410e11e"
server_key_version: "encryptionkey@0"
I20260806 06:31:24.456779 14211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/ts-1/wal/instance:
uuid: "98c66aa965b54ff79c1ff9677e6e94ba"
format_stamp: "Formatted at 2026-08-06 06:31:24 on dist-test-slave-6zbq"
server_key: "2860bd804e1fa5cda4beb8073a3f749f"
server_key_iv: "f276b8e8b4d02193b3ed84fa2410e11e"
server_key_version: "encryptionkey@0"
I20260806 06:31:24.457917 14211 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:31:24.458570 14230 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:24.458722 14211 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:24.458801 14211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/ts-1/wal
uuid: "98c66aa965b54ff79c1ff9677e6e94ba"
format_stamp: "Formatted at 2026-08-06 06:31:24 on dist-test-slave-6zbq"
server_key: "2860bd804e1fa5cda4beb8073a3f749f"
server_key_iv: "f276b8e8b4d02193b3ed84fa2410e11e"
server_key_version: "encryptionkey@0"
I20260806 06:31:24.458861 14211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:24.461973 14211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:24.462234 14211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:24.462378 14211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:24.462566 14211 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:31:24.462852 14211 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:31:24.462885 14211 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:24.462910 14211 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:31:24.462935 14211 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:24.468226 14211 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.66:37859
I20260806 06:31:24.468291 14400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.66:37859 every 8 connection(s)
I20260806 06:31:24.468552 14211 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
I20260806 06:31:24.469796 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 14211
I20260806 06:31:24.469884 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/ts-1/wal/instance
I20260806 06:31:24.470039 7341 external_mini_cluster.cc:1428] Setting key 024a97aa64358fe78e94922d10155eb5
I20260806 06:31:24.470997 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.67:0
--local_ip_for_outbound_sockets=127.7.43.67
--webserver_interface=127.7.43.67
--webserver_port=0
--tserver_master_addrs=127.7.43.126:43045
--builtin_ntp_servers=127.7.43.84:38139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260806 06:31:24.471413 13977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:24.469169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:50235 (local address 127.7.43.126:43045)
0806 06:31:24.469226 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:24.469228 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:24.469408 (+ 180us) server_negotiation.cc:407] Connection header received
0806 06:31:24.469863 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:24.469865 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:24.469871 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:24.469932 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:24.471158 (+ 1226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:24.471161 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:24.471161 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:31:24.471166 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:24.471182 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:31:24.471186 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:24.471215 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:24.471326 (+ 111us) server_negotiation.cc:299] Negotiation successful
0806 06:31:24.471345 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 06:31:24.471515 14403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:24.469125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:43045 (local address 127.7.43.66:50235)
0806 06:31:24.469299 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:24.469335 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:24.469760 (+ 425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:24.470003 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:24.470007 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:24.470017 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:24.471006 (+ 989us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:31:24.471006 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:24.471033 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:31:24.471060 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:24.471250 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:24.471254 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:24.471256 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:31:24.471309 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 06:31:24.471355 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":86,"threads_started":1}
I20260806 06:31:24.471890 14401 heartbeater.cc:347] Connected to a master server at 127.7.43.126:43045
I20260806 06:31:24.471974 14401 heartbeater.cc:464] Registering TS with master...
I20260806 06:31:24.472107 14401 heartbeater.cc:511] Master 127.7.43.126:43045 requested a full tablet report, sending...
I20260806 06:31:24.472471 13927 ts_manager.cc:194] Registered new tserver with Master: 98c66aa965b54ff79c1ff9677e6e94ba (127.7.43.66:37859)
W20260806 06:31:24.550930 14404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:24.551106 14404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:24.551127 14404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:24.552309 14404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:24.552350 14404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:24.552376 14404 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.67
I20260806 06:31:24.553697 14404 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:38139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.43.67
--webserver_port=0
--tserver_master_addrs=127.7.43.126:43045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.67
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:24.553901 14404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:24.554153 14404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:24.558687 14418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:24.558694 14413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:24.558843 14412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:24.558878 14404 server_base.cc:1034] running on GCE node
I20260806 06:31:24.559123 14404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:24.559310 14404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:24.560446 14404 hybrid_clock.cc:648] HybridClock initialized: now 1785997884560421 us; error 41 us; skew 500 ppm
I20260806 06:31:24.561606 14404 webserver.cc:460] Webserver started at http://127.7.43.67:33127/ using document root <none> and password file <none>
I20260806 06:31:24.561842 14404 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:24.561887 14404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:24.562009 14404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:24.563064 14404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/ts-2/data/instance:
uuid: "68941ba9419d48059d10e894578d3ba5"
format_stamp: "Formatted at 2026-08-06 06:31:24 on dist-test-slave-6zbq"
server_key: "42793cf86468937e6949b5a1dddc23cd"
server_key_iv: "1978b7a2300427780dc17b252485de77"
server_key_version: "encryptionkey@0"
I20260806 06:31:24.563388 14404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/ts-2/wal/instance:
uuid: "68941ba9419d48059d10e894578d3ba5"
format_stamp: "Formatted at 2026-08-06 06:31:24 on dist-test-slave-6zbq"
server_key: "42793cf86468937e6949b5a1dddc23cd"
server_key_iv: "1978b7a2300427780dc17b252485de77"
server_key_version: "encryptionkey@0"
I20260806 06:31:24.564493 14404 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:31:24.565088 14424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:24.565228 14404 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:31:24.565299 14404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/ts-2/wal
uuid: "68941ba9419d48059d10e894578d3ba5"
format_stamp: "Formatted at 2026-08-06 06:31:24 on dist-test-slave-6zbq"
server_key: "42793cf86468937e6949b5a1dddc23cd"
server_key_iv: "1978b7a2300427780dc17b252485de77"
server_key_version: "encryptionkey@0"
I20260806 06:31:24.565351 14404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:24.568446 14404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:24.568640 14404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:24.568727 14404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:24.568889 14404 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:31:24.569113 14404 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:31:24.569141 14404 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:24.569159 14404 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:31:24.569176 14404 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:24.574394 14404 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.67:41995
I20260806 06:31:24.574445 14603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.67:41995 every 8 connection(s)
I20260806 06:31:24.574666 14404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
I20260806 06:31:24.575937 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 14404
I20260806 06:31:24.576037 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/ts-2/wal/instance
I20260806 06:31:24.576203 7341 external_mini_cluster.cc:1428] Setting key 685316d24e42b95443639f8bf7f609e7
I20260806 06:31:24.577353 13977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:24.575260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:44059 (local address 127.7.43.126:43045)
0806 06:31:24.575330 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:24.575332 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:24.575507 (+ 175us) server_negotiation.cc:407] Connection header received
0806 06:31:24.575900 (+ 393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:24.575902 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:24.575909 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:24.575980 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:24.577134 (+ 1154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:24.577135 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:24.577135 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:31:24.577139 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:24.577154 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:31:24.577158 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:24.577203 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:24.577271 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 06:31:24.577285 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 06:31:24.577533 14606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:24.575227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:43045 (local address 127.7.43.67:44059)
0806 06:31:24.575407 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:24.575438 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:24.575816 (+ 378us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:24.576041 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:24.576045 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:24.576054 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:24.576994 (+ 940us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:31:24.576994 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:24.577029 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:31:24.577059 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:24.577209 (+ 150us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:24.577214 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:24.577216 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:31:24.577282 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 06:31:24.577338 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":93,"threads_started":1}
I20260806 06:31:24.577916 14604 heartbeater.cc:347] Connected to a master server at 127.7.43.126:43045
I20260806 06:31:24.577987 14604 heartbeater.cc:464] Registering TS with master...
I20260806 06:31:24.578088 14604 heartbeater.cc:511] Master 127.7.43.126:43045 requested a full tablet report, sending...
I20260806 06:31:24.578400 13927 ts_manager.cc:194] Registered new tserver with Master: 68941ba9419d48059d10e894578d3ba5 (127.7.43.67:41995)
I20260806 06:31:24.587706 7341 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:31:24.589861 13977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:24.588894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54876 (local address 127.7.43.126:43045)
0806 06:31:24.588948 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:24.588950 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:24.589036 (+ 86us) server_negotiation.cc:407] Connection header received
0806 06:31:24.589099 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:24.589100 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:24.589104 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:24.589156 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:24.589661 (+ 505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:24.589662 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:24.589663 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:31:24.589667 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:24.589679 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:31:24.589683 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:24.589728 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:24.589783 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 06:31:24.589794 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 06:31:24.591207 13927 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54876:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:31:24.592638 13927 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:31:24.596282 14623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:24.594800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45994 (local address 127.7.43.67:41995)
0806 06:31:24.594964 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:24.594968 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:24.595087 (+ 119us) server_negotiation.cc:407] Connection header received
0806 06:31:24.595134 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:24.595136 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:24.595145 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:24.595221 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:24.595942 (+ 721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:24.595949 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:24.595949 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:31:24.595954 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:24.595986 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:31:24.596006 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:24.596042 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:24.596194 (+ 152us) server_negotiation.cc:299] Negotiation successful
0806 06:31:24.596209 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":90,"threads_started":1}
I20260806 06:31:24.596288 14620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:24.594593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.67:41995 (local address 127.0.0.1:45994)
0806 06:31:24.595037 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:24.595045 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:24.595088 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:24.595489 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:24.595490 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:24.595492 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:24.595839 (+ 347us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:31:24.595840 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:24.595846 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:31:24.595856 (+ 10us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:24.596087 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:24.596093 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:24.596096 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:31:24.596178 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 06:31:24.596191 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"mutex_wait_us":12,"thread_start_us":84,"threads_started":1}
I20260806 06:31:24.596590 14624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:24.594848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42668 (local address 127.7.43.65:34161)
0806 06:31:24.595033 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:24.595036 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:24.595259 (+ 223us) server_negotiation.cc:407] Connection header received
0806 06:31:24.595290 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:24.595292 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:24.595301 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:24.595484 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:24.596234 (+ 750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:24.596244 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:24.596244 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:31:24.596248 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:24.596284 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:31:24.596314 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:24.596382 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:24.596493 (+ 111us) server_negotiation.cc:299] Negotiation successful
0806 06:31:24.596507 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":74,"threads_started":1}
I20260806 06:31:24.596971 14497 tablet_service.cc:1467] Processing CreateTablet for tablet a2a385bbd993478a8c4f98f9a0906f4e (DEFAULT_TABLE table=test-table [id=b31d698d56af44969a14a8d43ee2a0b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:31:24.597219 14497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2a385bbd993478a8c4f98f9a0906f4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:24.598526 14622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:24.594673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48902 (local address 127.7.43.66:37859)
0806 06:31:24.595001 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:24.595006 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:24.595016 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:31:24.595041 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:24.595044 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:24.595054 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:24.595128 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:24.596033 (+ 905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:24.596037 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:24.596037 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:31:24.596041 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:24.596070 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:31:24.596083 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:24.596139 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:24.598423 (+ 2284us) server_negotiation.cc:299] Negotiation successful
0806 06:31:24.598442 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":86,"threads_started":1}
I20260806 06:31:24.596306 14619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:24.594592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.66:37859 (local address 127.0.0.1:48902)
0806 06:31:24.594864 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:24.594901 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:24.594958 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:24.595140 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:24.595146 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:24.595153 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:24.595588 (+ 435us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:31:24.595589 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:24.595603 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:31:24.595623 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:24.596165 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:24.596166 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:24.596166 (+ 0us) client_negotiation.cc:769] Sending connection context
0806 06:31:24.596209 (+ 43us) client_negotiation.cc:240] Negotiation successful
0806 06:31:24.596222 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":65,"threads_started":1}
I20260806 06:31:24.598730 14617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:24.594609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:34161 (local address 127.0.0.1:42668)
0806 06:31:24.595163 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:24.595171 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:24.595206 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:24.595752 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:24.595753 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:24.595755 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:24.596131 (+ 376us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:31:24.596131 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:24.596138 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:31:24.596150 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:24.596410 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:24.596411 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:24.596412 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 06:31:24.596452 (+ 40us) client_negotiation.cc:240] Negotiation successful
0806 06:31:24.596463 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":82,"threads_started":1}
I20260806 06:31:24.599339 14114 tablet_service.cc:1467] Processing CreateTablet for tablet a2a385bbd993478a8c4f98f9a0906f4e (DEFAULT_TABLE table=test-table [id=b31d698d56af44969a14a8d43ee2a0b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:31:24.599304 14300 tablet_service.cc:1467] Processing CreateTablet for tablet a2a385bbd993478a8c4f98f9a0906f4e (DEFAULT_TABLE table=test-table [id=b31d698d56af44969a14a8d43ee2a0b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:31:24.599627 14114 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2a385bbd993478a8c4f98f9a0906f4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:24.599635 14300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2a385bbd993478a8c4f98f9a0906f4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:24.600049 14625 tablet_bootstrap.cc:492] T a2a385bbd993478a8c4f98f9a0906f4e P 68941ba9419d48059d10e894578d3ba5: Bootstrap starting.
I20260806 06:31:24.600734 14625 tablet_bootstrap.cc:654] T a2a385bbd993478a8c4f98f9a0906f4e P 68941ba9419d48059d10e894578d3ba5: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:24.601042 14625 log.cc:824] T a2a385bbd993478a8c4f98f9a0906f4e P 68941ba9419d48059d10e894578d3ba5: Log is configured to *not* fsync() on all Append() calls
I20260806 06:31:24.601809 14625 tablet_bootstrap.cc:492] T a2a385bbd993478a8c4f98f9a0906f4e P 68941ba9419d48059d10e894578d3ba5: No bootstrap required, opened a new log
I20260806 06:31:24.601878 14625 ts_tablet_manager.cc:1397] T a2a385bbd993478a8c4f98f9a0906f4e P 68941ba9419d48059d10e894578d3ba5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:31:24.602248 14629 tablet_bootstrap.cc:492] T a2a385bbd993478a8c4f98f9a0906f4e P 98c66aa965b54ff79c1ff9677e6e94ba: Bootstrap starting.
I20260806 06:31:24.602802 14630 tablet_bootstrap.cc:492] T a2a385bbd993478a8c4f98f9a0906f4e P eeb602dd23a94eb5a8d4b3e76b784c0d: Bootstrap starting.
I20260806 06:31:24.602972 14629 tablet_bootstrap.cc:654] T a2a385bbd993478a8c4f98f9a0906f4e P 98c66aa965b54ff79c1ff9677e6e94ba: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:24.603286 14629 log.cc:824] T a2a385bbd993478a8c4f98f9a0906f4e P 98c66aa965b54ff79c1ff9677e6e94ba: Log is configured to *not* fsync() on all Append() calls
I20260806 06:31:24.603456 14630 tablet_bootstrap.cc:654] T a2a385bbd993478a8c4f98f9a0906f4e P eeb602dd23a94eb5a8d4b3e76b784c0d: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:24.603762 14630 log.cc:824] T a2a385bbd993478a8c4f98f9a0906f4e P eeb602dd23a94eb5a8d4b3e76b784c0d: Log is configured to *not* fsync() on all Append() calls
I20260806 06:31:24.603976 14629 tablet_bootstrap.cc:492] T a2a385bbd993478a8c4f98f9a0906f4e P 98c66aa965b54ff79c1ff9677e6e94ba: No bootstrap required, opened a new log
I20260806 06:31:24.604030 14629 ts_tablet_manager.cc:1397] T a2a385bbd993478a8c4f98f9a0906f4e P 98c66aa965b54ff79c1ff9677e6e94ba: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:31:24.604449 14630 tablet_bootstrap.cc:492] T a2a385bbd993478a8c4f98f9a0906f4e P eeb602dd23a94eb5a8d4b3e76b784c0d: No bootstrap required, opened a new log
I20260806 06:31:24.604504 14630 ts_tablet_manager.cc:1397] T a2a385bbd993478a8c4f98f9a0906f4e P eeb602dd23a94eb5a8d4b3e76b784c0d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:31:24.604580 14625 raft_consensus.cc:348] T a2a385bbd993478a8c4f98f9a0906f4e P 68941ba9419d48059d10e894578d3ba5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98c66aa965b54ff79c1ff9677e6e94ba" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37859 } } peers { permanent_uuid: "68941ba9419d48059d10e894578d3ba5" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 41995 } } peers { permanent_uuid: "eeb602dd23a94eb5a8d4b3e76b784c0d" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 34161 } }
I20260806 06:31:24.604733 14625 raft_consensus.cc:374] T a2a385bbd993478a8c4f98f9a0906f4e P 68941ba9419d48059d10e894578d3ba5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:24.604764 14625 raft_consensus.cc:729] T a2a385bbd993478a8c4f98f9a0906f4e P 68941ba9419d48059d10e894578d3ba5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68941ba9419d48059d10e894578d3ba5, State: Initialized, Role: FOLLOWER
I20260806 06:31:24.604894 14625 consensus_queue.cc:260] T a2a385bbd993478a8c4f98f9a0906f4e P 68941ba9419d48059d10e894578d3ba5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98c66aa965b54ff79c1ff9677e6e94ba" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37859 } } peers { permanent_uuid: "68941ba9419d48059d10e894578d3ba5" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 41995 } } peers { permanent_uuid: "eeb602dd23a94eb5a8d4b3e76b784c0d" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 34161 } }
I20260806 06:31:24.605147 14625 ts_tablet_manager.cc:1428] T a2a385bbd993478a8c4f98f9a0906f4e P 68941ba9419d48059d10e894578d3ba5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:31:24.605247 14604 heartbeater.cc:503] Master 127.7.43.126:43045 was elected leader, sending a full tablet report...
I20260806 06:31:24.606647 14629 raft_consensus.cc:348] T a2a385bbd993478a8c4f98f9a0906f4e P 98c66aa965b54ff79c1ff9677e6e94ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98c66aa965b54ff79c1ff9677e6e94ba" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37859 } } peers { permanent_uuid: "68941ba9419d48059d10e894578d3ba5" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 41995 } } peers { permanent_uuid: "eeb602dd23a94eb5a8d4b3e76b784c0d" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 34161 } }
I20260806 06:31:24.606807 14629 raft_consensus.cc:374] T a2a385bbd993478a8c4f98f9a0906f4e P 98c66aa965b54ff79c1ff9677e6e94ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:24.606863 14629 raft_consensus.cc:729] T a2a385bbd993478a8c4f98f9a0906f4e P 98c66aa965b54ff79c1ff9677e6e94ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98c66aa965b54ff79c1ff9677e6e94ba, State: Initialized, Role: FOLLOWER
I20260806 06:31:24.606977 14629 consensus_queue.cc:260] T a2a385bbd993478a8c4f98f9a0906f4e P 98c66aa965b54ff79c1ff9677e6e94ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98c66aa965b54ff79c1ff9677e6e94ba" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37859 } } peers { permanent_uuid: "68941ba9419d48059d10e894578d3ba5" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 41995 } } peers { permanent_uuid: "eeb602dd23a94eb5a8d4b3e76b784c0d" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 34161 } }
I20260806 06:31:24.606928 14630 raft_consensus.cc:348] T a2a385bbd993478a8c4f98f9a0906f4e P eeb602dd23a94eb5a8d4b3e76b784c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98c66aa965b54ff79c1ff9677e6e94ba" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37859 } } peers { permanent_uuid: "68941ba9419d48059d10e894578d3ba5" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 41995 } } peers { permanent_uuid: "eeb602dd23a94eb5a8d4b3e76b784c0d" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 34161 } }
I20260806 06:31:24.607057 14630 raft_consensus.cc:374] T a2a385bbd993478a8c4f98f9a0906f4e P eeb602dd23a94eb5a8d4b3e76b784c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:24.607101 14630 raft_consensus.cc:729] T a2a385bbd993478a8c4f98f9a0906f4e P eeb602dd23a94eb5a8d4b3e76b784c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eeb602dd23a94eb5a8d4b3e76b784c0d, State: Initialized, Role: FOLLOWER
I20260806 06:31:24.607220 14629 ts_tablet_manager.cc:1428] T a2a385bbd993478a8c4f98f9a0906f4e P 98c66aa965b54ff79c1ff9677e6e94ba: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:31:24.607206 14630 consensus_queue.cc:260] T a2a385bbd993478a8c4f98f9a0906f4e P eeb602dd23a94eb5a8d4b3e76b784c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98c66aa965b54ff79c1ff9677e6e94ba" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37859 } } peers { permanent_uuid: "68941ba9419d48059d10e894578d3ba5" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 41995 } } peers { permanent_uuid: "eeb602dd23a94eb5a8d4b3e76b784c0d" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 34161 } }
I20260806 06:31:24.607259 14401 heartbeater.cc:503] Master 127.7.43.126:43045 was elected leader, sending a full tablet report...
I20260806 06:31:24.607455 14630 ts_tablet_manager.cc:1428] T a2a385bbd993478a8c4f98f9a0906f4e P eeb602dd23a94eb5a8d4b3e76b784c0d: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 06:31:24.607586 14208 heartbeater.cc:503] Master 127.7.43.126:43045 was elected leader, sending a full tablet report...
W20260806 06:31:24.719048 14402 tablet.cc:2369] T a2a385bbd993478a8c4f98f9a0906f4e P 98c66aa965b54ff79c1ff9677e6e94ba: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:31:24.798528 14636 raft_consensus.cc:482] T a2a385bbd993478a8c4f98f9a0906f4e P 68941ba9419d48059d10e894578d3ba5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:31:24.798647 14636 raft_consensus.cc:504] T a2a385bbd993478a8c4f98f9a0906f4e P 68941ba9419d48059d10e894578d3ba5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98c66aa965b54ff79c1ff9677e6e94ba" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37859 } } peers { permanent_uuid: "68941ba9419d48059d10e894578d3ba5" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 41995 } } peers { permanent_uuid: "eeb602dd23a94eb5a8d4b3e76b784c0d" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 34161 } }
I20260806 06:31:24.798969 14636 leader_election.cc:288] T a2a385bbd993478a8c4f98f9a0906f4e P 68941ba9419d48059d10e894578d3ba5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98c66aa965b54ff79c1ff9677e6e94ba (127.7.43.66:37859), eeb602dd23a94eb5a8d4b3e76b784c0d (127.7.43.65:34161)
I20260806 06:31:24.800130 14606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:24.799136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.66:37859 (local address 127.7.43.67:37809)
0806 06:31:24.799207 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:24.799217 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:24.799257 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:24.799441 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:24.799442 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:24.799445 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:24.799836 (+ 391us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:31:24.799836 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:24.799844 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:31:24.799856 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:24.800020 (+ 164us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:24.800021 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:24.800021 (+ 0us) client_negotiation.cc:769] Sending connection context
0806 06:31:24.800066 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 06:31:24.800077 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260806 06:31:24.800158 14622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:24.799212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:37809 (local address 127.7.43.66:37859)
0806 06:31:24.799253 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:24.799255 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:24.799263 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:31:24.799366 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:24.799367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:24.799374 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:24.799428 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:24.799934 (+ 506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:24.799935 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:24.799936 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:31:24.799941 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:24.799959 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:31:24.799963 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:24.800014 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:24.800072 (+ 58us) server_negotiation.cc:299] Negotiation successful
0806 06:31:24.800087 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 06:31:24.800436 14624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:24.799254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:41125 (local address 127.7.43.65:34161)
0806 06:31:24.799299 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:24.799301 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:24.799398 (+ 97us) server_negotiation.cc:407] Connection header received
0806 06:31:24.799424 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:24.799425 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:24.799429 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:24.799484 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:24.800058 (+ 574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:24.800060 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:24.800060 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:31:24.800064 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:24.800082 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:31:24.800087 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:24.800138 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:24.800350 (+ 212us) server_negotiation.cc:299] Negotiation successful
0806 06:31:24.800370 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 06:31:24.800457 14326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2a385bbd993478a8c4f98f9a0906f4e" candidate_uuid: "68941ba9419d48059d10e894578d3ba5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98c66aa965b54ff79c1ff9677e6e94ba" is_pre_election: true
I20260806 06:31:24.800597 14326 raft_consensus.cc:2454] T a2a385bbd993478a8c4f98f9a0906f4e P 98c66aa965b54ff79c1ff9677e6e94ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68941ba9419d48059d10e894578d3ba5 in term 0.
I20260806 06:31:24.800618 14640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:24.799139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:34161 (local address 127.7.43.67:41125)
0806 06:31:24.799333 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:24.799342 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:24.799383 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:24.799493 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:24.799494 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:24.799495 (+ 1us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:24.799965 (+ 470us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:31:24.799966 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:24.799972 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:31:24.799980 (+ 8us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:24.800140 (+ 160us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:24.800141 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:24.800141 (+ 0us) client_negotiation.cc:769] Sending connection context
0806 06:31:24.800530 (+ 389us) client_negotiation.cc:240] Negotiation successful
0806 06:31:24.800544 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"mutex_wait_us":12,"thread_start_us":110,"threads_started":1}
I20260806 06:31:24.800841 14432 leader_election.cc:302] T a2a385bbd993478a8c4f98f9a0906f4e P 68941ba9419d48059d10e894578d3ba5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68941ba9419d48059d10e894578d3ba5, 98c66aa965b54ff79c1ff9677e6e94ba; no voters:
I20260806 06:31:24.800941 14145 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2a385bbd993478a8c4f98f9a0906f4e" candidate_uuid: "68941ba9419d48059d10e894578d3ba5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eeb602dd23a94eb5a8d4b3e76b784c0d" is_pre_election: true
I20260806 06:31:24.800973 14636 raft_consensus.cc:2781] T a2a385bbd993478a8c4f98f9a0906f4e P 68941ba9419d48059d10e894578d3ba5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:31:24.801034 14636 raft_consensus.cc:482] T a2a385bbd993478a8c4f98f9a0906f4e P 68941ba9419d48059d10e894578d3ba5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:31:24.801050 14636 raft_consensus.cc:3037] T a2a385bbd993478a8c4f98f9a0906f4e P 68941ba9419d48059d10e894578d3ba5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:24.801084 14145 raft_consensus.cc:2454] T a2a385bbd993478a8c4f98f9a0906f4e P eeb602dd23a94eb5a8d4b3e76b784c0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68941ba9419d48059d10e894578d3ba5 in term 0.
I20260806 06:31:24.801812 14636 raft_consensus.cc:504] T a2a385bbd993478a8c4f98f9a0906f4e P 68941ba9419d48059d10e894578d3ba5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98c66aa965b54ff79c1ff9677e6e94ba" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37859 } } peers { permanent_uuid: "68941ba9419d48059d10e894578d3ba5" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 41995 } } peers { permanent_uuid: "eeb602dd23a94eb5a8d4b3e76b784c0d" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 34161 } }
I20260806 06:31:24.802003 14636 leader_election.cc:288] T a2a385bbd993478a8c4f98f9a0906f4e P 68941ba9419d48059d10e894578d3ba5 [CANDIDATE]: Term 1 election: Requested vote from peers 98c66aa965b54ff79c1ff9677e6e94ba (127.7.43.66:37859), eeb602dd23a94eb5a8d4b3e76b784c0d (127.7.43.65:34161)
I20260806 06:31:24.802084 14326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2a385bbd993478a8c4f98f9a0906f4e" candidate_uuid: "68941ba9419d48059d10e894578d3ba5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98c66aa965b54ff79c1ff9677e6e94ba"
I20260806 06:31:24.802124 14145 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2a385bbd993478a8c4f98f9a0906f4e" candidate_uuid: "68941ba9419d48059d10e894578d3ba5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eeb602dd23a94eb5a8d4b3e76b784c0d"
I20260806 06:31:24.802167 14326 raft_consensus.cc:3037] T a2a385bbd993478a8c4f98f9a0906f4e P 98c66aa965b54ff79c1ff9677e6e94ba [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:24.802193 14145 raft_consensus.cc:3037] T a2a385bbd993478a8c4f98f9a0906f4e P eeb602dd23a94eb5a8d4b3e76b784c0d [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:24.802882 14326 raft_consensus.cc:2454] T a2a385bbd993478a8c4f98f9a0906f4e P 98c66aa965b54ff79c1ff9677e6e94ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68941ba9419d48059d10e894578d3ba5 in term 1.
I20260806 06:31:24.802901 14145 raft_consensus.cc:2454] T a2a385bbd993478a8c4f98f9a0906f4e P eeb602dd23a94eb5a8d4b3e76b784c0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68941ba9419d48059d10e894578d3ba5 in term 1.
I20260806 06:31:24.803035 14432 leader_election.cc:302] T a2a385bbd993478a8c4f98f9a0906f4e P 68941ba9419d48059d10e894578d3ba5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68941ba9419d48059d10e894578d3ba5, 98c66aa965b54ff79c1ff9677e6e94ba; no voters:
I20260806 06:31:24.803153 14636 raft_consensus.cc:2781] T a2a385bbd993478a8c4f98f9a0906f4e P 68941ba9419d48059d10e894578d3ba5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:31:24.803304 14636 raft_consensus.cc:686] T a2a385bbd993478a8c4f98f9a0906f4e P 68941ba9419d48059d10e894578d3ba5 [term 1 LEADER]: Becoming Leader. State: Replica: 68941ba9419d48059d10e894578d3ba5, State: Running, Role: LEADER
I20260806 06:31:24.803432 14636 consensus_queue.cc:237] T a2a385bbd993478a8c4f98f9a0906f4e P 68941ba9419d48059d10e894578d3ba5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98c66aa965b54ff79c1ff9677e6e94ba" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37859 } } peers { permanent_uuid: "68941ba9419d48059d10e894578d3ba5" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 41995 } } peers { permanent_uuid: "eeb602dd23a94eb5a8d4b3e76b784c0d" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 34161 } }
I20260806 06:31:24.804170 13925 catalog_manager.cc:5495] T a2a385bbd993478a8c4f98f9a0906f4e P 68941ba9419d48059d10e894578d3ba5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 68941ba9419d48059d10e894578d3ba5 (127.7.43.67). New cstate: current_term: 1 leader_uuid: "68941ba9419d48059d10e894578d3ba5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98c66aa965b54ff79c1ff9677e6e94ba" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68941ba9419d48059d10e894578d3ba5" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 41995 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eeb602dd23a94eb5a8d4b3e76b784c0d" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 34161 } health_report { overall_health: UNKNOWN } } }
W20260806 06:31:24.825126 14605 tablet.cc:2369] T a2a385bbd993478a8c4f98f9a0906f4e P 68941ba9419d48059d10e894578d3ba5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:31:24.827047 14623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:24.826247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46000 (local address 127.7.43.67:41995)
0806 06:31:24.826298 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:24.826299 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:24.826306 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:31:24.826349 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:24.826350 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:24.826363 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:24.826411 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:24.826853 (+ 442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:24.826854 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:24.826854 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:31:24.826858 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:24.826871 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:31:24.826875 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:24.826924 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:24.826975 (+ 51us) server_negotiation.cc:299] Negotiation successful
0806 06:31:24.826986 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 06:31:24.828650 14326 raft_consensus.cc:1261] T a2a385bbd993478a8c4f98f9a0906f4e P 98c66aa965b54ff79c1ff9677e6e94ba [term 1 FOLLOWER]: Refusing update from remote peer 68941ba9419d48059d10e894578d3ba5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:31:24.828923 14145 raft_consensus.cc:1261] T a2a385bbd993478a8c4f98f9a0906f4e P eeb602dd23a94eb5a8d4b3e76b784c0d [term 1 FOLLOWER]: Refusing update from remote peer 68941ba9419d48059d10e894578d3ba5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:31:24.828986 14641 consensus_queue.cc:1035] T a2a385bbd993478a8c4f98f9a0906f4e P 68941ba9419d48059d10e894578d3ba5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98c66aa965b54ff79c1ff9677e6e94ba" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:31:24.829169 14636 consensus_queue.cc:1035] T a2a385bbd993478a8c4f98f9a0906f4e P 68941ba9419d48059d10e894578d3ba5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eeb602dd23a94eb5a8d4b3e76b784c0d" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 34161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:31:24.831127 14644 mvcc.cc:205] Tried to move back new op lower bound from 7315447336255619072 to 7315447336155291648. Current Snapshot: MvccSnapshot[applied={T|T < 7315447336255619072}]
I20260806 06:31:24.831810 14647 mvcc.cc:205] Tried to move back new op lower bound from 7315447336255619072 to 7315447336155291648. Current Snapshot: MvccSnapshot[applied={T|T < 7315447336255619072 or (T in {7315447336255619072})}]
I20260806 06:31:24.833117 13927 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54876:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:31:24.833212 13927 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54876:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:31:24.833935 13927 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fade6d4ef3bd4bc790b118e0dc805a3a: Sending DeleteTablet for 3 replicas of tablet a2a385bbd993478a8c4f98f9a0906f4e
I20260806 06:31:24.834242 14497 tablet_service.cc:1514] Processing DeleteTablet for tablet a2a385bbd993478a8c4f98f9a0906f4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:31:24 UTC) from {username='slave'} at 127.0.0.1:45994
I20260806 06:31:24.834242 14300 tablet_service.cc:1514] Processing DeleteTablet for tablet a2a385bbd993478a8c4f98f9a0906f4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:31:24 UTC) from {username='slave'} at 127.0.0.1:48902
I20260806 06:31:24.834285 14114 tablet_service.cc:1514] Processing DeleteTablet for tablet a2a385bbd993478a8c4f98f9a0906f4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:31:24 UTC) from {username='slave'} at 127.0.0.1:42668
I20260806 06:31:24.834522 14661 tablet_replica.cc:330] T a2a385bbd993478a8c4f98f9a0906f4e P 68941ba9419d48059d10e894578d3ba5: stopping tablet replica
I20260806 06:31:24.834621 14661 raft_consensus.cc:2229] T a2a385bbd993478a8c4f98f9a0906f4e P 68941ba9419d48059d10e894578d3ba5 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:31:24.834712 14661 raft_consensus.cc:2258] T a2a385bbd993478a8c4f98f9a0906f4e P 68941ba9419d48059d10e894578d3ba5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:31:24.834724 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 14010
I20260806 06:31:24.841823 14661 ts_tablet_manager.cc:1905] T a2a385bbd993478a8c4f98f9a0906f4e P 68941ba9419d48059d10e894578d3ba5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:31:24.842468 14662 tablet_replica.cc:330] T a2a385bbd993478a8c4f98f9a0906f4e P 98c66aa965b54ff79c1ff9677e6e94ba: stopping tablet replica
I20260806 06:31:24.842567 14662 raft_consensus.cc:2229] T a2a385bbd993478a8c4f98f9a0906f4e P 98c66aa965b54ff79c1ff9677e6e94ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:31:24.842672 14662 raft_consensus.cc:2258] T a2a385bbd993478a8c4f98f9a0906f4e P 98c66aa965b54ff79c1ff9677e6e94ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:31:24.843111 14662 ts_tablet_manager.cc:1905] T a2a385bbd993478a8c4f98f9a0906f4e P 98c66aa965b54ff79c1ff9677e6e94ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:31:24.843458 14661 ts_tablet_manager.cc:1918] T a2a385bbd993478a8c4f98f9a0906f4e P 68941ba9419d48059d10e894578d3ba5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:31:24.843537 14661 log.cc:1196] T a2a385bbd993478a8c4f98f9a0906f4e P 68941ba9419d48059d10e894578d3ba5: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/ts-2/wal/wals/a2a385bbd993478a8c4f98f9a0906f4e
I20260806 06:31:24.843806 14661 ts_tablet_manager.cc:1939] T a2a385bbd993478a8c4f98f9a0906f4e P 68941ba9419d48059d10e894578d3ba5: Deleting consensus metadata
I20260806 06:31:24.844170 13903 catalog_manager.cc:4841] TS 68941ba9419d48059d10e894578d3ba5 (127.7.43.67:41995): tablet a2a385bbd993478a8c4f98f9a0906f4e (table test-table [id=b31d698d56af44969a14a8d43ee2a0b6]) successfully deleted
I20260806 06:31:24.844873 14662 ts_tablet_manager.cc:1918] T a2a385bbd993478a8c4f98f9a0906f4e P 98c66aa965b54ff79c1ff9677e6e94ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:31:24.845005 14662 log.cc:1196] T a2a385bbd993478a8c4f98f9a0906f4e P 98c66aa965b54ff79c1ff9677e6e94ba: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785997861828271-7341-0/minicluster-data/ts-1/wal/wals/a2a385bbd993478a8c4f98f9a0906f4e
I20260806 06:31:24.845291 14662 ts_tablet_manager.cc:1939] T a2a385bbd993478a8c4f98f9a0906f4e P 98c66aa965b54ff79c1ff9677e6e94ba: Deleting consensus metadata
I20260806 06:31:24.845731 13906 catalog_manager.cc:4841] TS 98c66aa965b54ff79c1ff9677e6e94ba (127.7.43.66:37859): tablet a2a385bbd993478a8c4f98f9a0906f4e (table test-table [id=b31d698d56af44969a14a8d43ee2a0b6]) successfully deleted
W20260806 06:31:24.847361 13905 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.43.65:34161 (error 108)
I20260806 06:31:24.847527 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 14211
I20260806 06:31:24.858803 14617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:24.858663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:34161 (local address 127.0.0.1:42682)
0806 06:31:24.858712 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:24.858763 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.65:34161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260806 06:31:24.858964 13905 catalog_manager.cc:4584] TS eeb602dd23a94eb5a8d4b3e76b784c0d (127.7.43.65:34161): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a2a385bbd993478a8c4f98f9a0906f4e: Network error: Client connection negotiation failed: client connection to 127.7.43.65:34161: connect: Connection refused (error 111)
I20260806 06:31:24.859107 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 14404
I20260806 06:31:24.867614 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 13882
2026-08-06T06:31:24Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (796 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260806 06:31:24.890309 7341 mini_oidc.cc:152] Starting JWKS server
I20260806 06:31:24.891551 7341 webserver.cc:460] Webserver started at https://127.0.0.1:36223/ using document root <none> and password file <none>
I20260806 06:31:24.891631 7341 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 06:31:24.891781 7341 webserver.cc:460] Webserver started at http://127.0.0.1:46345/ using document root <none> and password file <none>
2026-08-06T06:31:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:31:24Z Disabled control of system clock
I20260806 06:31:24.900235 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:39281
--webserver_interface=127.7.43.126
--webserver_port=0
--builtin_ntp_servers=127.7.43.84:33647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.43.126:39281
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36223/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785997861828271-7341-0/testchainca.cert with env {}
W20260806 06:31:24.977806 14672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:24.977978 14672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:24.977998 14672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:24.979130 14672 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:31:24.979166 14672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:24.979176 14672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:24.979188 14672 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:31:24.979211 14672 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 06:31:24.979236 14672 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 06:31:24.979257 14672 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36223/jwks.json
I20260806 06:31:24.980939 14672 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:33647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785997861828271-7341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.43.126:39281
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:39281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36223/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.43.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785997861828271-7341-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14672
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:24.981172 14672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:24.981377 14672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:24.985996 14678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:24.986058 14677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:24.986006 14681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:24.986243 14672 server_base.cc:1034] running on GCE node
I20260806 06:31:24.986475 14672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:24.986721 14672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:24.987855 14672 hybrid_clock.cc:648] HybridClock initialized: now 1785997884987840 us; error 25 us; skew 500 ppm
I20260806 06:31:24.988898 14672 webserver.cc:460] Webserver started at http://127.7.43.126:33873/ using document root <none> and password file <none>
I20260806 06:31:24.989099 14672 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:24.989145 14672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:24.989271 14672 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:24.990172 14672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785997861828271-7341-0/minicluster-data/master-0/data/instance:
uuid: "9ed05366858844c7b91dfaf5e200c866"
format_stamp: "Formatted at 2026-08-06 06:31:24 on dist-test-slave-6zbq"
server_key: "708b93e433eda7528ac30590d48c22f9"
server_key_iv: "fd40e22ab1bbc29db99a3aaeb7496ef5"
server_key_version: "encryptionkey@0"
I20260806 06:31:24.990511 14672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785997861828271-7341-0/minicluster-data/master-0/wal/instance:
uuid: "9ed05366858844c7b91dfaf5e200c866"
format_stamp: "Formatted at 2026-08-06 06:31:24 on dist-test-slave-6zbq"
server_key: "708b93e433eda7528ac30590d48c22f9"
server_key_iv: "fd40e22ab1bbc29db99a3aaeb7496ef5"
server_key_version: "encryptionkey@0"
I20260806 06:31:24.991621 14672 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:31:24.992211 14691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:24.992363 14672 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:31:24.992429 14672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "9ed05366858844c7b91dfaf5e200c866"
format_stamp: "Formatted at 2026-08-06 06:31:24 on dist-test-slave-6zbq"
server_key: "708b93e433eda7528ac30590d48c22f9"
server_key_iv: "fd40e22ab1bbc29db99a3aaeb7496ef5"
server_key_version: "encryptionkey@0"
I20260806 06:31:24.992498 14672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:25.013978 14672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:25.014211 14672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:25.014331 14672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:25.017817 14672 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.126:39281
I20260806 06:31:25.017867 14774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.126:39281 every 8 connection(s)
I20260806 06:31:25.018139 14672 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:31:25.019060 14775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:25.021061 14775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ed05366858844c7b91dfaf5e200c866: Bootstrap starting.
I20260806 06:31:25.021602 14775 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ed05366858844c7b91dfaf5e200c866: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:25.021860 14775 log.cc:824] T 00000000000000000000000000000000 P 9ed05366858844c7b91dfaf5e200c866: Log is configured to *not* fsync() on all Append() calls
I20260806 06:31:25.022475 14775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ed05366858844c7b91dfaf5e200c866: No bootstrap required, opened a new log
I20260806 06:31:25.023950 14775 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ed05366858844c7b91dfaf5e200c866 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ed05366858844c7b91dfaf5e200c866" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 39281 } }
I20260806 06:31:25.024071 14775 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9ed05366858844c7b91dfaf5e200c866 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:25.024102 14775 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ed05366858844c7b91dfaf5e200c866 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ed05366858844c7b91dfaf5e200c866, State: Initialized, Role: FOLLOWER
I20260806 06:31:25.024227 14775 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ed05366858844c7b91dfaf5e200c866 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ed05366858844c7b91dfaf5e200c866" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 39281 } }
I20260806 06:31:25.024286 14775 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9ed05366858844c7b91dfaf5e200c866 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:31:25.024307 14775 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9ed05366858844c7b91dfaf5e200c866 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:31:25.024351 14775 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ed05366858844c7b91dfaf5e200c866 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:25.024544 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 14672
I20260806 06:31:25.024644 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785997861828271-7341-0/minicluster-data/master-0/wal/instance
I20260806 06:31:25.024822 7341 external_mini_cluster.cc:1428] Setting key 5aa1b9ce19c78d78a0e92fbafea608d3
I20260806 06:31:25.024969 14775 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9ed05366858844c7b91dfaf5e200c866 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ed05366858844c7b91dfaf5e200c866" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 39281 } }
I20260806 06:31:25.025061 14775 leader_election.cc:302] T 00000000000000000000000000000000 P 9ed05366858844c7b91dfaf5e200c866 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ed05366858844c7b91dfaf5e200c866; no voters:
I20260806 06:31:25.025218 14775 leader_election.cc:288] T 00000000000000000000000000000000 P 9ed05366858844c7b91dfaf5e200c866 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:31:25.025262 14779 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9ed05366858844c7b91dfaf5e200c866 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:31:25.025375 14779 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9ed05366858844c7b91dfaf5e200c866 [term 1 LEADER]: Becoming Leader. State: Replica: 9ed05366858844c7b91dfaf5e200c866, State: Running, Role: LEADER
I20260806 06:31:25.025450 14775 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ed05366858844c7b91dfaf5e200c866 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:31:25.025442 14779 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ed05366858844c7b91dfaf5e200c866 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ed05366858844c7b91dfaf5e200c866" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 39281 } }
I20260806 06:31:25.025815 14781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ed05366858844c7b91dfaf5e200c866 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9ed05366858844c7b91dfaf5e200c866" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ed05366858844c7b91dfaf5e200c866" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 39281 } } }
I20260806 06:31:25.025911 14781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ed05366858844c7b91dfaf5e200c866 [sys.catalog]: This master's current role is: LEADER
I20260806 06:31:25.026086 14782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ed05366858844c7b91dfaf5e200c866 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ed05366858844c7b91dfaf5e200c866. Latest consensus state: current_term: 1 leader_uuid: "9ed05366858844c7b91dfaf5e200c866" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ed05366858844c7b91dfaf5e200c866" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 39281 } } }
I20260806 06:31:25.026149 14782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ed05366858844c7b91dfaf5e200c866 [sys.catalog]: This master's current role is: LEADER
I20260806 06:31:25.026304 14792 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:31:25.026813 14792 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:31:25.028860 14792 catalog_manager.cc:1347] Generated new cluster ID: 59fd6eb54de74d6381ab71ef9f69cd4f
I20260806 06:31:25.028915 14792 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:31:25.030881 14783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:25.025288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39494 (local address 127.7.43.126:39281)
0806 06:31:25.025643 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:25.025653 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:25.025684 (+ 31us) server_negotiation.cc:407] Connection header received
0806 06:31:25.025798 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:25.025805 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:25.025874 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:25.026474 (+ 600us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:25.026585 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:25.027894 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:25.028802 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:25.029413 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:25.029529 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:25.030175 (+ 646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:25.030178 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:25.030178 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:31:25.030411 (+ 233us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:25.030452 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:31:25.030465 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:25.030523 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:25.030612 (+ 89us) server_negotiation.cc:299] Negotiation successful
0806 06:31:25.030682 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":94,"threads_started":1}
I20260806 06:31:25.031577 7341 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:31:25.035943 14792 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:31:25.036585 14792 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:31:25.039462 14792 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9ed05366858844c7b91dfaf5e200c866: Generated new TSK 0
I20260806 06:31:25.039602 14792 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 06:31:25.072242 14814 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260806 06:31:25.072350 14814 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 06:31:25.072582 14783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:25.049843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39510 (local address 127.7.43.126:39281)
0806 06:31:25.049885 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:25.049887 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:25.050005 (+ 118us) server_negotiation.cc:407] Connection header received
0806 06:31:25.050099 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:25.050099 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:25.050111 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:25.050161 (+ 50us) server_negotiation.cc:226] Negotiated authn=JWT
0806 06:31:25.050312 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:25.050786 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:25.051517 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:25.051701 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:25.051752 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:25.051947 (+ 195us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 06:31:25.072390 (+ 20443us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36223/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 06:31:25.072520 (+ 130us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39510: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36223/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":19}
W20260806 06:31:25.072645 14783 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39510: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36223/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260806 06:31:25.072916 14813 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:25.049773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:39281 (local address 127.0.0.1:39510)
0806 06:31:25.049944 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:25.049953 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:25.050010 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:25.050176 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:25.050177 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:25.050179 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0806 06:31:25.050255 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:25.050257 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:25.050874 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:25.050875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:25.051439 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:25.051441 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:25.051766 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:25.051767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:25.051827 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:25.051848 (+ 21us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 06:31:25.072489 (+ 20641us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36223/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 06:31:25.072573 (+ 84us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.43.126:39281: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36223/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":71,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.7.43.126:39281: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36223/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260806 06:31:25.073292 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 14672
2026-08-06T06:31:25Z chronyd exiting
I20260806 06:31:25.114959 7341 test_util.cc:175] -----------------------------------------------
I20260806 06:31:25.115021 7341 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785997861828271-7341-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (225 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260806 06:31:25.115754 7341 mini_oidc.cc:152] Starting JWKS server
I20260806 06:31:25.116387 7341 webserver.cc:460] Webserver started at https://127.0.0.1:40027/ using document root <none> and password file <none>
I20260806 06:31:25.116427 7341 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 06:31:25.116561 7341 webserver.cc:460] Webserver started at http://127.0.0.1:38991/ using document root <none> and password file <none>
2026-08-06T06:31:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:31:25Z Disabled control of system clock
I20260806 06:31:25.124490 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:38279
--webserver_interface=127.7.43.126
--webserver_port=0
--builtin_ntp_servers=127.7.43.84:41003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.43.126:38279
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40027/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785997861828271-7341-0/testchainca.cert with env {}
W20260806 06:31:25.203085 14831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:25.203263 14831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:25.203284 14831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:25.204370 14831 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:31:25.204406 14831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:25.204416 14831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:25.204427 14831 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:31:25.204444 14831 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 06:31:25.204455 14831 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 06:31:25.204465 14831 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40027/jwks.json
I20260806 06:31:25.205886 14831 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:41003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785997861828271-7341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.43.126:38279
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:38279
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40027/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.43.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785997861828271-7341-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14831
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:25.206063 14831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:25.206214 14831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:25.210574 14844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:25.210575 14839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:25.210698 14831 server_base.cc:1034] running on GCE node
W20260806 06:31:25.210601 14841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:25.210999 14831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:25.211275 14831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:25.212417 14831 hybrid_clock.cc:648] HybridClock initialized: now 1785997885212404 us; error 29 us; skew 500 ppm
I20260806 06:31:25.217350 14831 webserver.cc:460] Webserver started at http://127.7.43.126:35301/ using document root <none> and password file <none>
I20260806 06:31:25.217540 14831 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:25.217581 14831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:25.217691 14831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:25.218708 14831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785997861828271-7341-0/minicluster-data/master-0/data/instance:
uuid: "d9475eae1bd64407a202bb6a93d18b0e"
format_stamp: "Formatted at 2026-08-06 06:31:25 on dist-test-slave-6zbq"
server_key: "f0835952adf14df923c0184f8e85a893"
server_key_iv: "1dc1116f8a2e5f6b1a1760878fc4a160"
server_key_version: "encryptionkey@0"
I20260806 06:31:25.219015 14831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785997861828271-7341-0/minicluster-data/master-0/wal/instance:
uuid: "d9475eae1bd64407a202bb6a93d18b0e"
format_stamp: "Formatted at 2026-08-06 06:31:25 on dist-test-slave-6zbq"
server_key: "f0835952adf14df923c0184f8e85a893"
server_key_iv: "1dc1116f8a2e5f6b1a1760878fc4a160"
server_key_version: "encryptionkey@0"
I20260806 06:31:25.220189 14831 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:31:25.220815 14852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:25.220956 14831 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:31:25.221032 14831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "d9475eae1bd64407a202bb6a93d18b0e"
format_stamp: "Formatted at 2026-08-06 06:31:25 on dist-test-slave-6zbq"
server_key: "f0835952adf14df923c0184f8e85a893"
server_key_iv: "1dc1116f8a2e5f6b1a1760878fc4a160"
server_key_version: "encryptionkey@0"
I20260806 06:31:25.221107 14831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:25.228956 14831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:25.229184 14831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:25.229293 14831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:25.232844 14831 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.126:38279
I20260806 06:31:25.232872 14926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.126:38279 every 8 connection(s)
I20260806 06:31:25.233150 14831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:31:25.233960 14927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:25.236035 14927 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9475eae1bd64407a202bb6a93d18b0e: Bootstrap starting.
I20260806 06:31:25.236555 14927 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d9475eae1bd64407a202bb6a93d18b0e: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:25.236799 14927 log.cc:824] T 00000000000000000000000000000000 P d9475eae1bd64407a202bb6a93d18b0e: Log is configured to *not* fsync() on all Append() calls
I20260806 06:31:25.237411 14927 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9475eae1bd64407a202bb6a93d18b0e: No bootstrap required, opened a new log
I20260806 06:31:25.238318 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 14831
I20260806 06:31:25.238427 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785997861828271-7341-0/minicluster-data/master-0/wal/instance
I20260806 06:31:25.238579 7341 external_mini_cluster.cc:1428] Setting key daa9737887db67d309ea3265a4af82b9
I20260806 06:31:25.239053 14927 raft_consensus.cc:348] T 00000000000000000000000000000000 P d9475eae1bd64407a202bb6a93d18b0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9475eae1bd64407a202bb6a93d18b0e" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 38279 } }
I20260806 06:31:25.239217 14927 raft_consensus.cc:374] T 00000000000000000000000000000000 P d9475eae1bd64407a202bb6a93d18b0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:25.239269 14927 raft_consensus.cc:729] T 00000000000000000000000000000000 P d9475eae1bd64407a202bb6a93d18b0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9475eae1bd64407a202bb6a93d18b0e, State: Initialized, Role: FOLLOWER
I20260806 06:31:25.239392 14927 consensus_queue.cc:260] T 00000000000000000000000000000000 P d9475eae1bd64407a202bb6a93d18b0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9475eae1bd64407a202bb6a93d18b0e" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 38279 } }
I20260806 06:31:25.239462 14927 raft_consensus.cc:388] T 00000000000000000000000000000000 P d9475eae1bd64407a202bb6a93d18b0e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:31:25.239483 14927 raft_consensus.cc:482] T 00000000000000000000000000000000 P d9475eae1bd64407a202bb6a93d18b0e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:31:25.239527 14927 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d9475eae1bd64407a202bb6a93d18b0e [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:25.240005 14927 raft_consensus.cc:504] T 00000000000000000000000000000000 P d9475eae1bd64407a202bb6a93d18b0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9475eae1bd64407a202bb6a93d18b0e" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 38279 } }
I20260806 06:31:25.240100 14927 leader_election.cc:302] T 00000000000000000000000000000000 P d9475eae1bd64407a202bb6a93d18b0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d9475eae1bd64407a202bb6a93d18b0e; no voters:
I20260806 06:31:25.240259 14927 leader_election.cc:288] T 00000000000000000000000000000000 P d9475eae1bd64407a202bb6a93d18b0e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:31:25.240432 14935 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d9475eae1bd64407a202bb6a93d18b0e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:31:25.240494 14927 sys_catalog.cc:564] T 00000000000000000000000000000000 P d9475eae1bd64407a202bb6a93d18b0e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:31:25.240564 14935 raft_consensus.cc:686] T 00000000000000000000000000000000 P d9475eae1bd64407a202bb6a93d18b0e [term 1 LEADER]: Becoming Leader. State: Replica: d9475eae1bd64407a202bb6a93d18b0e, State: Running, Role: LEADER
I20260806 06:31:25.240671 14935 consensus_queue.cc:237] T 00000000000000000000000000000000 P d9475eae1bd64407a202bb6a93d18b0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9475eae1bd64407a202bb6a93d18b0e" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 38279 } }
I20260806 06:31:25.240968 14935 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9475eae1bd64407a202bb6a93d18b0e [sys.catalog]: SysCatalogTable state changed. Reason: New leader d9475eae1bd64407a202bb6a93d18b0e. Latest consensus state: current_term: 1 leader_uuid: "d9475eae1bd64407a202bb6a93d18b0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9475eae1bd64407a202bb6a93d18b0e" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 38279 } } }
I20260806 06:31:25.241035 14935 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9475eae1bd64407a202bb6a93d18b0e [sys.catalog]: This master's current role is: LEADER
I20260806 06:31:25.241182 14939 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9475eae1bd64407a202bb6a93d18b0e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d9475eae1bd64407a202bb6a93d18b0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9475eae1bd64407a202bb6a93d18b0e" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 38279 } } }
I20260806 06:31:25.241246 14939 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9475eae1bd64407a202bb6a93d18b0e [sys.catalog]: This master's current role is: LEADER
I20260806 06:31:25.242033 14948 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:31:25.242708 14948 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:31:25.243989 14933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:25.239023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37576 (local address 127.7.43.126:38279)
0806 06:31:25.239207 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:25.239215 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:25.239246 (+ 31us) server_negotiation.cc:407] Connection header received
0806 06:31:25.239370 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:25.239375 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:25.239425 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:25.239548 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:25.239849 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:25.241058 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:25.242019 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:25.242550 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:25.242659 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:25.243268 (+ 609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:25.243273 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:25.243273 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:31:25.243465 (+ 192us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:25.243505 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:31:25.243519 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:25.243587 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:25.243686 (+ 99us) server_negotiation.cc:299] Negotiation successful
0806 06:31:25.243751 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":71,"threads_started":1}
I20260806 06:31:25.244539 14948 catalog_manager.cc:1347] Generated new cluster ID: 6f6e6896838542de8627bff12a556a3a
I20260806 06:31:25.244585 14948 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:31:25.244941 7341 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:31:25.270213 14948 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:31:25.270937 14948 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:31:25.281692 14948 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d9475eae1bd64407a202bb6a93d18b0e: Generated new TSK 0
I20260806 06:31:25.281849 14948 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 06:31:25.299656 14971 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260806 06:31:25.299708 14971 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 06:31:25.300006 14933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:25.286276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37588 (local address 127.7.43.126:38279)
0806 06:31:25.286318 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:25.286320 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:25.286423 (+ 103us) server_negotiation.cc:407] Connection header received
0806 06:31:25.286469 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:25.286470 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:25.286481 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:25.286560 (+ 79us) server_negotiation.cc:226] Negotiated authn=JWT
0806 06:31:25.286649 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:25.287401 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:25.288122 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:25.288440 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:25.288508 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:25.288627 (+ 119us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 06:31:25.299824 (+ 11197us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40027/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 06:31:25.299949 (+ 125us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37588: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40027/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":24}
W20260806 06:31:25.300056 14933 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37588: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40027/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260806 06:31:25.300077 14970 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:25.286209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38279 (local address 127.0.0.1:37588)
0806 06:31:25.286328 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:25.286364 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:25.286422 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:25.286558 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:25.286558 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:25.286560 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0806 06:31:25.286603 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:25.286605 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:25.287482 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:25.287482 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:25.288044 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:25.288046 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:25.288508 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:25.288509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:25.288545 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:25.288549 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 06:31:25.299918 (+ 11369us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40027/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 06:31:25.299999 (+ 81us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.43.126:38279: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40027/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":74,"threads_started":1}
I20260806 06:31:25.300323 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 14831
2026-08-06T06:31:25Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (227 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260806 06:31:25.343484 7341 mini_oidc.cc:152] Starting JWKS server
I20260806 06:31:25.344113 7341 webserver.cc:460] Webserver started at https://127.0.0.1:36511/ using document root <none> and password file <none>
I20260806 06:31:25.344159 7341 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 06:31:25.344290 7341 webserver.cc:460] Webserver started at http://127.0.0.1:43183/ using document root <none> and password file <none>
2026-08-06T06:31:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:31:25Z Disabled control of system clock
I20260806 06:31:25.353539 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:33287
--webserver_interface=127.7.43.126
--webserver_port=0
--builtin_ntp_servers=127.7.43.84:43891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.43.126:33287
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36511/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 06:31:25.431627 14978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:25.431807 14978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:25.431826 14978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:25.432942 14978 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:31:25.432986 14978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:25.432996 14978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:25.433007 14978 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:31:25.433018 14978 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 06:31:25.433029 14978 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 06:31:25.433039 14978 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36511/jwks.json
I20260806 06:31:25.434405 14978 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:43891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785997861828271-7341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.43.126:33287
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:33287
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36511/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.43.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14978
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:25.434620 14978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:25.434878 14978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:25.439214 14990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:25.439222 14987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:25.439254 14978 server_base.cc:1034] running on GCE node
W20260806 06:31:25.439230 14992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:25.439641 14978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:25.439919 14978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:25.441054 14978 hybrid_clock.cc:648] HybridClock initialized: now 1785997885441041 us; error 29 us; skew 500 ppm
I20260806 06:31:25.442137 14978 webserver.cc:460] Webserver started at http://127.7.43.126:42093/ using document root <none> and password file <none>
I20260806 06:31:25.442373 14978 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:25.442427 14978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:25.442549 14978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:25.443496 14978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785997861828271-7341-0/minicluster-data/master-0/data/instance:
uuid: "d8170cd0ef794740b53c7f9a78876791"
format_stamp: "Formatted at 2026-08-06 06:31:25 on dist-test-slave-6zbq"
server_key: "7a0e3da213f3c936c424afc7ef5ae80a"
server_key_iv: "83f46d22f61afa987fac1243ca0df285"
server_key_version: "encryptionkey@0"
I20260806 06:31:25.443838 14978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785997861828271-7341-0/minicluster-data/master-0/wal/instance:
uuid: "d8170cd0ef794740b53c7f9a78876791"
format_stamp: "Formatted at 2026-08-06 06:31:25 on dist-test-slave-6zbq"
server_key: "7a0e3da213f3c936c424afc7ef5ae80a"
server_key_iv: "83f46d22f61afa987fac1243ca0df285"
server_key_version: "encryptionkey@0"
I20260806 06:31:25.445013 14978 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:31:25.445637 15000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:25.445791 14978 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:31:25.445870 14978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "d8170cd0ef794740b53c7f9a78876791"
format_stamp: "Formatted at 2026-08-06 06:31:25 on dist-test-slave-6zbq"
server_key: "7a0e3da213f3c936c424afc7ef5ae80a"
server_key_iv: "83f46d22f61afa987fac1243ca0df285"
server_key_version: "encryptionkey@0"
I20260806 06:31:25.445922 14978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:25.457681 14978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:25.457894 14978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:25.457983 14978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:25.461432 14978 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.126:33287
I20260806 06:31:25.461468 15081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.126:33287 every 8 connection(s)
I20260806 06:31:25.461735 14978 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:31:25.462378 15083 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:25.464959 15083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8170cd0ef794740b53c7f9a78876791: Bootstrap starting.
I20260806 06:31:25.465493 15083 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d8170cd0ef794740b53c7f9a78876791: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:25.465755 15083 log.cc:824] T 00000000000000000000000000000000 P d8170cd0ef794740b53c7f9a78876791: Log is configured to *not* fsync() on all Append() calls
I20260806 06:31:25.466393 15083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8170cd0ef794740b53c7f9a78876791: No bootstrap required, opened a new log
I20260806 06:31:25.467418 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 14978
I20260806 06:31:25.467533 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785997861828271-7341-0/minicluster-data/master-0/wal/instance
I20260806 06:31:25.467716 7341 external_mini_cluster.cc:1428] Setting key 5024178839d9e31cee0e85edc570c220
I20260806 06:31:25.468178 15083 raft_consensus.cc:348] T 00000000000000000000000000000000 P d8170cd0ef794740b53c7f9a78876791 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8170cd0ef794740b53c7f9a78876791" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 33287 } }
I20260806 06:31:25.468344 15083 raft_consensus.cc:374] T 00000000000000000000000000000000 P d8170cd0ef794740b53c7f9a78876791 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:25.468371 15083 raft_consensus.cc:729] T 00000000000000000000000000000000 P d8170cd0ef794740b53c7f9a78876791 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8170cd0ef794740b53c7f9a78876791, State: Initialized, Role: FOLLOWER
I20260806 06:31:25.468490 15083 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8170cd0ef794740b53c7f9a78876791 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8170cd0ef794740b53c7f9a78876791" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 33287 } }
I20260806 06:31:25.468554 15083 raft_consensus.cc:388] T 00000000000000000000000000000000 P d8170cd0ef794740b53c7f9a78876791 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:31:25.468577 15083 raft_consensus.cc:482] T 00000000000000000000000000000000 P d8170cd0ef794740b53c7f9a78876791 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:31:25.468621 15083 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d8170cd0ef794740b53c7f9a78876791 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:25.469422 15083 raft_consensus.cc:504] T 00000000000000000000000000000000 P d8170cd0ef794740b53c7f9a78876791 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8170cd0ef794740b53c7f9a78876791" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 33287 } }
I20260806 06:31:25.469509 15083 leader_election.cc:302] T 00000000000000000000000000000000 P d8170cd0ef794740b53c7f9a78876791 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d8170cd0ef794740b53c7f9a78876791; no voters:
I20260806 06:31:25.469659 15083 leader_election.cc:288] T 00000000000000000000000000000000 P d8170cd0ef794740b53c7f9a78876791 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:31:25.469702 15091 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d8170cd0ef794740b53c7f9a78876791 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:31:25.469822 15091 raft_consensus.cc:686] T 00000000000000000000000000000000 P d8170cd0ef794740b53c7f9a78876791 [term 1 LEADER]: Becoming Leader. State: Replica: d8170cd0ef794740b53c7f9a78876791, State: Running, Role: LEADER
I20260806 06:31:25.469905 15083 sys_catalog.cc:564] T 00000000000000000000000000000000 P d8170cd0ef794740b53c7f9a78876791 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:31:25.469903 15091 consensus_queue.cc:237] T 00000000000000000000000000000000 P d8170cd0ef794740b53c7f9a78876791 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8170cd0ef794740b53c7f9a78876791" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 33287 } }
I20260806 06:31:25.471088 15093 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8170cd0ef794740b53c7f9a78876791 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8170cd0ef794740b53c7f9a78876791. Latest consensus state: current_term: 1 leader_uuid: "d8170cd0ef794740b53c7f9a78876791" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8170cd0ef794740b53c7f9a78876791" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 33287 } } }
I20260806 06:31:25.471184 15093 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8170cd0ef794740b53c7f9a78876791 [sys.catalog]: This master's current role is: LEADER
I20260806 06:31:25.471231 15094 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8170cd0ef794740b53c7f9a78876791 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d8170cd0ef794740b53c7f9a78876791" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8170cd0ef794740b53c7f9a78876791" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 33287 } } }
I20260806 06:31:25.471462 15094 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8170cd0ef794740b53c7f9a78876791 [sys.catalog]: This master's current role is: LEADER
I20260806 06:31:25.471390 15113 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:31:25.471915 15113 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:31:25.473438 15113 catalog_manager.cc:1347] Generated new cluster ID: 46a586fefe6941139dcdcab827602973
I20260806 06:31:25.473484 15090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:25.468163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53508 (local address 127.7.43.126:33287)
0806 06:31:25.468387 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:25.468395 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:25.468426 (+ 31us) server_negotiation.cc:407] Connection header received
0806 06:31:25.468522 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:25.468527 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:25.468587 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:25.468712 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:25.468928 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:25.470241 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:25.471314 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:25.471811 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:25.471931 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:25.472806 (+ 875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:25.472810 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:25.472810 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:31:25.472985 (+ 175us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:25.473024 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:31:25.473039 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:25.473108 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:25.473205 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 06:31:25.473282 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":97,"threads_started":1}
I20260806 06:31:25.473490 15113 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:31:25.474282 7341 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:31:25.499913 15113 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:31:25.500674 15113 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:31:25.505014 15113 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d8170cd0ef794740b53c7f9a78876791: Generated new TSK 0
I20260806 06:31:25.505187 15113 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:31:25.513748 15090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:25.510270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53520 (local address 127.7.43.126:33287)
0806 06:31:25.510319 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:25.510321 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:25.510418 (+ 97us) server_negotiation.cc:407] Connection header received
0806 06:31:25.510478 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:25.510479 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:25.510491 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:25.510577 (+ 86us) server_negotiation.cc:226] Negotiated authn=JWT
0806 06:31:25.510673 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:25.511443 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:25.512154 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:25.512452 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:25.512518 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:25.512635 (+ 117us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 06:31:25.513553 (+ 918us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36511/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 06:31:25.513676 (+ 123us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53520: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36511/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":28}
W20260806 06:31:25.513799 15090 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53520: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36511/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260806 06:31:25.515578 15124 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:25.510151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33287 (local address 127.0.0.1:53520)
0806 06:31:25.510317 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:25.510330 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:25.510423 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:25.510574 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:25.510575 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:25.510577 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0806 06:31:25.510628 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:25.510630 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:25.511516 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:25.511516 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:25.512077 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:25.512078 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:25.512514 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:25.512514 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:25.512549 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:25.512553 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 06:31:25.513653 (+ 1100us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36511/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 06:31:25.515476 (+ 1823us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.43.126:33287: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36511/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":1646,"thread_start_us":75,"threads_started":1}
I20260806 06:31:25.515767 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 14978
E20260806 06:31:25.533928 15125 webserver.cc:533] Webserver: sslize failed:
E20260806 06:31:25.533998 15125 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-06T06:31:25Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (219 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-06T06:31:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:31:25Z Disabled control of system clock
I20260806 06:31:25.570367 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:44489
--webserver_interface=127.7.43.126
--webserver_port=0
--builtin_ntp_servers=127.7.43.84:42063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.43.126:44489
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785997861828271-7341-0/testchainca.cert with env {}
W20260806 06:31:25.656659 15134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:25.656842 15134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:25.656872 15134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:25.657987 15134 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:31:25.658026 15134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:25.658043 15134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:25.658059 15134 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:31:25.658075 15134 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 06:31:25.658097 15134 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 06:31:25.658114 15134 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260806 06:31:25.659590 15134 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:42063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785997861828271-7341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.43.126:44489
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:44489
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.43.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785997861828271-7341-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:25.659828 15134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:25.660007 15134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:25.664470 15140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:25.664502 15143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:25.664480 15141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:25.664515 15134 server_base.cc:1034] running on GCE node
I20260806 06:31:25.664896 15134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:25.665150 15134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:25.666275 15134 hybrid_clock.cc:648] HybridClock initialized: now 1785997885666265 us; error 26 us; skew 500 ppm
I20260806 06:31:25.667348 15134 webserver.cc:460] Webserver started at http://127.7.43.126:38397/ using document root <none> and password file <none>
I20260806 06:31:25.667559 15134 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:25.667603 15134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:25.667718 15134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:25.668661 15134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785997861828271-7341-0/minicluster-data/master-0/data/instance:
uuid: "540be515e099435d9274e616915d74ba"
format_stamp: "Formatted at 2026-08-06 06:31:25 on dist-test-slave-6zbq"
server_key: "70ee73581d6aeb8e4da2ca549ec3e521"
server_key_iv: "5b70fee3b995f36eaeaca472a761c7e0"
server_key_version: "encryptionkey@0"
I20260806 06:31:25.668972 15134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785997861828271-7341-0/minicluster-data/master-0/wal/instance:
uuid: "540be515e099435d9274e616915d74ba"
format_stamp: "Formatted at 2026-08-06 06:31:25 on dist-test-slave-6zbq"
server_key: "70ee73581d6aeb8e4da2ca549ec3e521"
server_key_iv: "5b70fee3b995f36eaeaca472a761c7e0"
server_key_version: "encryptionkey@0"
I20260806 06:31:25.670135 15134 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:31:25.670837 15152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:25.670985 15134 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:25.671066 15134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "540be515e099435d9274e616915d74ba"
format_stamp: "Formatted at 2026-08-06 06:31:25 on dist-test-slave-6zbq"
server_key: "70ee73581d6aeb8e4da2ca549ec3e521"
server_key_iv: "5b70fee3b995f36eaeaca472a761c7e0"
server_key_version: "encryptionkey@0"
I20260806 06:31:25.671139 15134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:25.681499 15134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:25.681737 15134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:25.681856 15134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:25.685554 15134 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.126:44489
I20260806 06:31:25.685583 15230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.126:44489 every 8 connection(s)
I20260806 06:31:25.685866 15134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:31:25.686730 15232 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:25.688771 15232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 540be515e099435d9274e616915d74ba: Bootstrap starting.
I20260806 06:31:25.689347 15232 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 540be515e099435d9274e616915d74ba: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:25.689575 15232 log.cc:824] T 00000000000000000000000000000000 P 540be515e099435d9274e616915d74ba: Log is configured to *not* fsync() on all Append() calls
I20260806 06:31:25.690140 15232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 540be515e099435d9274e616915d74ba: No bootstrap required, opened a new log
I20260806 06:31:25.691689 15232 raft_consensus.cc:348] T 00000000000000000000000000000000 P 540be515e099435d9274e616915d74ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "540be515e099435d9274e616915d74ba" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 44489 } }
I20260806 06:31:25.691818 15232 raft_consensus.cc:374] T 00000000000000000000000000000000 P 540be515e099435d9274e616915d74ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:25.691839 15232 raft_consensus.cc:729] T 00000000000000000000000000000000 P 540be515e099435d9274e616915d74ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 540be515e099435d9274e616915d74ba, State: Initialized, Role: FOLLOWER
I20260806 06:31:25.691934 15232 consensus_queue.cc:260] T 00000000000000000000000000000000 P 540be515e099435d9274e616915d74ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "540be515e099435d9274e616915d74ba" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 44489 } }
I20260806 06:31:25.691982 15232 raft_consensus.cc:388] T 00000000000000000000000000000000 P 540be515e099435d9274e616915d74ba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:31:25.692000 15232 raft_consensus.cc:482] T 00000000000000000000000000000000 P 540be515e099435d9274e616915d74ba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:31:25.692026 15232 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 540be515e099435d9274e616915d74ba [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:25.692487 15232 raft_consensus.cc:504] T 00000000000000000000000000000000 P 540be515e099435d9274e616915d74ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "540be515e099435d9274e616915d74ba" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 44489 } }
I20260806 06:31:25.692569 15232 leader_election.cc:302] T 00000000000000000000000000000000 P 540be515e099435d9274e616915d74ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 540be515e099435d9274e616915d74ba; no voters:
I20260806 06:31:25.692674 15232 leader_election.cc:288] T 00000000000000000000000000000000 P 540be515e099435d9274e616915d74ba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:31:25.692734 15237 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 540be515e099435d9274e616915d74ba [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:31:25.692878 15232 sys_catalog.cc:564] T 00000000000000000000000000000000 P 540be515e099435d9274e616915d74ba [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:31:25.692883 15237 raft_consensus.cc:686] T 00000000000000000000000000000000 P 540be515e099435d9274e616915d74ba [term 1 LEADER]: Becoming Leader. State: Replica: 540be515e099435d9274e616915d74ba, State: Running, Role: LEADER
I20260806 06:31:25.693005 15237 consensus_queue.cc:237] T 00000000000000000000000000000000 P 540be515e099435d9274e616915d74ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "540be515e099435d9274e616915d74ba" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 44489 } }
I20260806 06:31:25.693364 15238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 540be515e099435d9274e616915d74ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "540be515e099435d9274e616915d74ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "540be515e099435d9274e616915d74ba" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 44489 } } }
I20260806 06:31:25.693454 15238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 540be515e099435d9274e616915d74ba [sys.catalog]: This master's current role is: LEADER
I20260806 06:31:25.693480 15239 sys_catalog.cc:455] T 00000000000000000000000000000000 P 540be515e099435d9274e616915d74ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 540be515e099435d9274e616915d74ba. Latest consensus state: current_term: 1 leader_uuid: "540be515e099435d9274e616915d74ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "540be515e099435d9274e616915d74ba" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 44489 } } }
I20260806 06:31:25.693562 15239 sys_catalog.cc:458] T 00000000000000000000000000000000 P 540be515e099435d9274e616915d74ba [sys.catalog]: This master's current role is: LEADER
I20260806 06:31:25.693697 15249 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:31:25.694173 15249 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:31:25.694515 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 15134
I20260806 06:31:25.694603 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785997861828271-7341-0/minicluster-data/master-0/wal/instance
I20260806 06:31:25.694778 7341 external_mini_cluster.cc:1428] Setting key 5ac459723740c1a46788e07eb4e9cf0b
I20260806 06:31:25.695717 15249 catalog_manager.cc:1347] Generated new cluster ID: 5f9ce602771d4ec3835ad3d5c1a518a9
I20260806 06:31:25.695770 15249 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:31:25.703312 15266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:25.695357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49616 (local address 127.7.43.126:44489)
0806 06:31:25.695550 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:25.695557 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:25.695620 (+ 63us) server_negotiation.cc:407] Connection header received
0806 06:31:25.698539 (+ 2919us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:25.698547 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:25.698618 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:25.698776 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:25.698936 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:25.700311 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:25.701302 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:25.701929 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:25.702045 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:25.702648 (+ 603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:25.702652 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:25.702653 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:31:25.702864 (+ 211us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:25.702904 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:31:25.702918 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:25.702983 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:25.703038 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 06:31:25.703110 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":82,"threads_started":1}
I20260806 06:31:25.704342 7341 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:31:25.706190 15249 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:31:25.706660 15249 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:31:25.710202 15249 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 540be515e099435d9274e616915d74ba: Generated new TSK 0
I20260806 06:31:25.710332 15249 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:31:25.752507 15266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:25.720118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49622 (local address 127.7.43.126:44489)
0806 06:31:25.720176 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:25.720177 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:25.720273 (+ 96us) server_negotiation.cc:407] Connection header received
0806 06:31:25.720338 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:25.720339 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:25.720348 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:25.720400 (+ 52us) server_negotiation.cc:226] Negotiated authn=JWT
0806 06:31:25.720514 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:25.720975 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:25.722027 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:25.722236 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:25.722285 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:25.722475 (+ 190us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 06:31:25.752330 (+ 29855us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 06:31:25.752458 (+ 128us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49622: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":27}
W20260806 06:31:25.752560 15266 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49622: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260806 06:31:25.752682 15275 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:25.720058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:44489 (local address 127.0.0.1:49622)
0806 06:31:25.720206 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:25.720218 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:25.720280 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:25.720411 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:25.720412 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:25.720413 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0806 06:31:25.720463 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:25.720465 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:25.721038 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:25.721039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:25.721949 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:25.721951 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:25.722299 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:25.722300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:25.722392 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:25.722397 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 06:31:25.752455 (+ 30058us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 06:31:25.752581 (+ 126us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.43.126:44489: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":71,"threads_started":1}
I20260806 06:31:25.753185 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 15134
2026-08-06T06:31:25Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (211 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (87 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (84 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:31:25 dist-test-slave-6zbq krb5kdc[15299](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:31:25 dist-test-slave-6zbq krb5kdc[15299](info): set up 2 sockets
Aug 06 06:31:25 dist-test-slave-6zbq krb5kdc[15299](info): commencing operation
krb5kdc: starting...
W20260806 06:31:27.029119 7341 mini_kdc.cc:121] Time spent starting KDC: real 1.083s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Aug 06 06:31:27 dist-test-slave-6zbq krb5kdc[15299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:31:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:31:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.43.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
I20260806 06:31:27.136210 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:38483
--webserver_interface=127.7.43.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:45307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.43.126:38483
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:31:27.213666 15317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:27.213850 15317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:27.213878 15317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:27.215023 15317 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:31:27.215063 15317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:27.215083 15317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:27.215098 15317 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:31:27.215114 15317 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:31:27.216516 15317 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:45307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.43.126:38483
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:38483
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.43.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15317
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:27.216741 15317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:27.216936 15317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:31:27.221328 15317 server_base.cc:1034] running on GCE node
W20260806 06:31:27.221302 15328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:27.221287 15324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:27.221302 15330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:27.221721 15317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:27.221977 15317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:27.223097 15317 hybrid_clock.cc:648] HybridClock initialized: now 1785997887223087 us; error 26 us; skew 500 ppm
Aug 06 06:31:27 dist-test-slave-6zbq krb5kdc[15299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997887, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:27.225237 15317 init.cc:377] Logged in from keytab as kudu/127.7.43.126@KRBTEST.COM (short username kudu)
I20260806 06:31:27.225625 15317 webserver.cc:460] Webserver started at http://127.7.43.126:34711/ using document root <none> and password file <none>
I20260806 06:31:27.225829 15317 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:27.225876 15317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:27.225994 15317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:27.226977 15317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/master-0/data/instance:
uuid: "914825175eb34178ac01b64535070c0c"
format_stamp: "Formatted at 2026-08-06 06:31:27 on dist-test-slave-6zbq"
server_key: "c6dcb1b7dccede38dacc4d2b031b9ce9"
server_key_iv: "bec17787c4756ba4abdf64bf22e3b49c"
server_key_version: "encryptionkey@0"
I20260806 06:31:27.227288 15317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/master-0/wal/instance:
uuid: "914825175eb34178ac01b64535070c0c"
format_stamp: "Formatted at 2026-08-06 06:31:27 on dist-test-slave-6zbq"
server_key: "c6dcb1b7dccede38dacc4d2b031b9ce9"
server_key_iv: "bec17787c4756ba4abdf64bf22e3b49c"
server_key_version: "encryptionkey@0"
I20260806 06:31:27.228410 15317 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:31:27.229005 15346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:27.229151 15317 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:31:27.229233 15317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "914825175eb34178ac01b64535070c0c"
format_stamp: "Formatted at 2026-08-06 06:31:27 on dist-test-slave-6zbq"
server_key: "c6dcb1b7dccede38dacc4d2b031b9ce9"
server_key_iv: "bec17787c4756ba4abdf64bf22e3b49c"
server_key_version: "encryptionkey@0"
I20260806 06:31:27.229290 15317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:27.245143 15317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:27.246969 15317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:27.247117 15317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:27.251729 15317 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.126:38483
I20260806 06:31:27.251811 15416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.126:38483 every 8 connection(s)
I20260806 06:31:27.252384 15317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:31:27.253456 15417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:27.255877 15417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 914825175eb34178ac01b64535070c0c: Bootstrap starting.
I20260806 06:31:27.256456 15417 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 914825175eb34178ac01b64535070c0c: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:27.256733 15417 log.cc:824] T 00000000000000000000000000000000 P 914825175eb34178ac01b64535070c0c: Log is configured to *not* fsync() on all Append() calls
I20260806 06:31:27.257457 15417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 914825175eb34178ac01b64535070c0c: No bootstrap required, opened a new log
I20260806 06:31:27.259116 15417 raft_consensus.cc:348] T 00000000000000000000000000000000 P 914825175eb34178ac01b64535070c0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "914825175eb34178ac01b64535070c0c" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 38483 } }
I20260806 06:31:27.259276 15417 raft_consensus.cc:374] T 00000000000000000000000000000000 P 914825175eb34178ac01b64535070c0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:27.259301 15417 raft_consensus.cc:729] T 00000000000000000000000000000000 P 914825175eb34178ac01b64535070c0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 914825175eb34178ac01b64535070c0c, State: Initialized, Role: FOLLOWER
I20260806 06:31:27.259433 15417 consensus_queue.cc:260] T 00000000000000000000000000000000 P 914825175eb34178ac01b64535070c0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "914825175eb34178ac01b64535070c0c" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 38483 } }
I20260806 06:31:27.259496 15417 raft_consensus.cc:388] T 00000000000000000000000000000000 P 914825175eb34178ac01b64535070c0c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:31:27.259516 15417 raft_consensus.cc:482] T 00000000000000000000000000000000 P 914825175eb34178ac01b64535070c0c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:31:27.259545 15417 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 914825175eb34178ac01b64535070c0c [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:27.260177 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 15317
I20260806 06:31:27.260293 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/master-0/wal/instance
I20260806 06:31:27.260483 7341 external_mini_cluster.cc:1428] Setting key ecf69b9df6e4f412f0e667012931b6c3
I20260806 06:31:27.260645 15417 raft_consensus.cc:504] T 00000000000000000000000000000000 P 914825175eb34178ac01b64535070c0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "914825175eb34178ac01b64535070c0c" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 38483 } }
I20260806 06:31:27.260754 15417 leader_election.cc:302] T 00000000000000000000000000000000 P 914825175eb34178ac01b64535070c0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 914825175eb34178ac01b64535070c0c; no voters:
I20260806 06:31:27.260918 15417 leader_election.cc:288] T 00000000000000000000000000000000 P 914825175eb34178ac01b64535070c0c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:31:27.261169 15420 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 914825175eb34178ac01b64535070c0c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:31:27.261212 15417 sys_catalog.cc:564] T 00000000000000000000000000000000 P 914825175eb34178ac01b64535070c0c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:31:27.261409 15420 raft_consensus.cc:686] T 00000000000000000000000000000000 P 914825175eb34178ac01b64535070c0c [term 1 LEADER]: Becoming Leader. State: Replica: 914825175eb34178ac01b64535070c0c, State: Running, Role: LEADER
I20260806 06:31:27.261497 15420 consensus_queue.cc:237] T 00000000000000000000000000000000 P 914825175eb34178ac01b64535070c0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "914825175eb34178ac01b64535070c0c" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 38483 } }
I20260806 06:31:27.261868 15425 sys_catalog.cc:455] T 00000000000000000000000000000000 P 914825175eb34178ac01b64535070c0c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 914825175eb34178ac01b64535070c0c. Latest consensus state: current_term: 1 leader_uuid: "914825175eb34178ac01b64535070c0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "914825175eb34178ac01b64535070c0c" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 38483 } } }
I20260806 06:31:27.262010 15425 sys_catalog.cc:458] T 00000000000000000000000000000000 P 914825175eb34178ac01b64535070c0c [sys.catalog]: This master's current role is: LEADER
I20260806 06:31:27.262205 15422 sys_catalog.cc:455] T 00000000000000000000000000000000 P 914825175eb34178ac01b64535070c0c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "914825175eb34178ac01b64535070c0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "914825175eb34178ac01b64535070c0c" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 38483 } } }
I20260806 06:31:27.262318 15422 sys_catalog.cc:458] T 00000000000000000000000000000000 P 914825175eb34178ac01b64535070c0c [sys.catalog]: This master's current role is: LEADER
W20260806 06:31:27.263224 15444 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 914825175eb34178ac01b64535070c0c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:31:27.263280 15444 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:31:27.263322 15436 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:31:27.263463 15436 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:31:27.265031 15436 catalog_manager.cc:1347] Generated new cluster ID: 648b7b6ba1644da6ad7036f8d71a513d
I20260806 06:31:27.265079 15436 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:31:27 dist-test-slave-6zbq krb5kdc[15299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:31:27.271394 15423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:27.260987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50058 (local address 127.7.43.126:38483)
0806 06:31:27.261174 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:27.261185 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:27.261221 (+ 36us) server_negotiation.cc:407] Connection header received
0806 06:31:27.261370 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:27.261378 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:27.261441 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:27.262404 (+ 963us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:27.262507 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:27.263575 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:27.264434 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:27.264955 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:27.265109 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:27.267289 (+ 2180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:27.267294 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:27.267294 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:27.267307 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:27.269745 (+ 2438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:27.270099 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:27.270102 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:27.270103 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:27.270197 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:27.270409 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:27.270410 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:27.270411 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:27.270651 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:27.270800 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:27.271111 (+ 311us) server_negotiation.cc:299] Negotiation successful
0806 06:31:27.271232 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":79,"threads_started":1}
I20260806 06:31:27.280001 15436 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:31:27.280479 15436 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:31:27.285091 15436 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 914825175eb34178ac01b64535070c0c: Generated new TSK 0
I20260806 06:31:27.285229 15436 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.43.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab.
Entry for principal kudu/127.7.43.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab.
I20260806 06:31:27.313170 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.65:0
--local_ip_for_outbound_sockets=127.7.43.65
--webserver_interface=127.7.43.65
--webserver_port=0
--tserver_master_addrs=127.7.43.126:38483
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab
--principal=kudu/127.7.43.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:45307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:31:27.393309 15449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:27.393504 15449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:27.393534 15449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:27.394695 15449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:27.394735 15449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:27.394768 15449 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.65
I20260806 06:31:27.396201 15449 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:45307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab
--principal=kudu/127.7.43.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.43.65
--webserver_port=0
--tserver_master_addrs=127.7.43.126:38483
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15449
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.65
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:27.396420 15449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:27.396613 15449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:27.401896 15457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:27.401901 15462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:27.401901 15458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:27.401933 15449 server_base.cc:1034] running on GCE node
I20260806 06:31:27.402300 15449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:27.402544 15449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:27.403728 15449 hybrid_clock.cc:648] HybridClock initialized: now 1785997887403714 us; error 35 us; skew 500 ppm
Aug 06 06:31:27 dist-test-slave-6zbq krb5kdc[15299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997887, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:27.406112 15449 init.cc:377] Logged in from keytab as kudu/127.7.43.65@KRBTEST.COM (short username kudu)
I20260806 06:31:27.406577 15449 webserver.cc:460] Webserver started at http://127.7.43.65:34001/ using document root <none> and password file <none>
I20260806 06:31:27.406788 15449 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:27.406836 15449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:27.406953 15449 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:27.407991 15449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/ts-0/data/instance:
uuid: "d321036f41c4433f97caa2c1afa3cf76"
format_stamp: "Formatted at 2026-08-06 06:31:27 on dist-test-slave-6zbq"
server_key: "6a770cf82cdb670acad011ce90037a4d"
server_key_iv: "88029753f297e6eafc6977d313b81400"
server_key_version: "encryptionkey@0"
I20260806 06:31:27.408316 15449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/ts-0/wal/instance:
uuid: "d321036f41c4433f97caa2c1afa3cf76"
format_stamp: "Formatted at 2026-08-06 06:31:27 on dist-test-slave-6zbq"
server_key: "6a770cf82cdb670acad011ce90037a4d"
server_key_iv: "88029753f297e6eafc6977d313b81400"
server_key_version: "encryptionkey@0"
I20260806 06:31:27.409539 15449 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 06:31:27.410194 15470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:27.410423 15449 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:31:27.410519 15449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/ts-0/wal
uuid: "d321036f41c4433f97caa2c1afa3cf76"
format_stamp: "Formatted at 2026-08-06 06:31:27 on dist-test-slave-6zbq"
server_key: "6a770cf82cdb670acad011ce90037a4d"
server_key_iv: "88029753f297e6eafc6977d313b81400"
server_key_version: "encryptionkey@0"
I20260806 06:31:27.410588 15449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:27.419028 15449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:27.420771 15449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:27.420893 15449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:27.421103 15449 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:31:27.421383 15449 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:31:27.421416 15449 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:27.421442 15449 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:31:27.421468 15449 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:27.427047 15449 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.65:33283
I20260806 06:31:27.427107 15635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.65:33283 every 8 connection(s)
I20260806 06:31:27.427366 15449 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
I20260806 06:31:27.428169 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 15449
I20260806 06:31:27.428272 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/ts-0/wal/instance
I20260806 06:31:27.428454 7341 external_mini_cluster.cc:1428] Setting key 405d26d206f14d20e0fa3be4ba295067
Aug 06 06:31:27 dist-test-slave-6zbq krb5kdc[15299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997887, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.65@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.43.66@KRBTEST.COM; defaulting to no policy
I20260806 06:31:27.436724 15423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:27.428019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:38039 (local address 127.7.43.126:38483)
0806 06:31:27.428077 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:27.428079 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:27.428297 (+ 218us) server_negotiation.cc:407] Connection header received
0806 06:31:27.429324 (+ 1027us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:27.429325 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:27.429337 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:27.429385 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:27.429993 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:27.430541 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:27.431571 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:27.431820 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:27.431884 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:27.434107 (+ 2223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:27.434109 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:27.434109 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:27.434118 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:27.435381 (+ 1263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:27.435846 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:27.435847 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:27.435848 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:27.435896 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:27.436146 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:27.436146 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:27.436147 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:27.436239 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:27.436282 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:27.436575 (+ 293us) server_negotiation.cc:299] Negotiation successful
0806 06:31:27.436658 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 06:31:27.436946 15641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:27.428000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38483 (local address 127.7.43.65:38039)
0806 06:31:27.428156 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:27.428191 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:27.429225 (+ 1034us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:27.429464 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:27.429468 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:27.429713 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:27.429897 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:27.429901 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:27.430623 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:27.430625 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:27.431492 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:27.431495 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:27.431904 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:27.431904 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:27.432024 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:27.432505 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:27.432513 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:27.434013 (+ 1500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:27.435462 (+ 1449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:27.435469 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:27.435472 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:27.435763 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:27.435951 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:27.435952 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:27.435952 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:27.436080 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:27.436292 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:27.436298 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:27.436450 (+ 152us) client_negotiation.cc:769] Sending connection context
0806 06:31:27.436560 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 06:31:27.436739 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":80,"threads_started":1}
I20260806 06:31:27.437417 15636 heartbeater.cc:347] Connected to a master server at 127.7.43.126:38483
I20260806 06:31:27.437511 15636 heartbeater.cc:464] Registering TS with master...
I20260806 06:31:27.437677 15636 heartbeater.cc:511] Master 127.7.43.126:38483 requested a full tablet report, sending...
I20260806 06:31:27.438146 15365 ts_manager.cc:194] Registered new tserver with Master: d321036f41c4433f97caa2c1afa3cf76 (127.7.43.65:33283)
I20260806 06:31:27.438822 15365 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.65@KRBTEST.COM'} at 127.7.43.65:38039
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab.
Entry for principal kudu/127.7.43.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab.
I20260806 06:31:27.450726 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.66:0
--local_ip_for_outbound_sockets=127.7.43.66
--webserver_interface=127.7.43.66
--webserver_port=0
--tserver_master_addrs=127.7.43.126:38483
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab
--principal=kudu/127.7.43.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:45307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:31:27.528615 15645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:27.528787 15645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:27.528808 15645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:27.529863 15645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:27.529894 15645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:27.529918 15645 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.66
I20260806 06:31:27.531337 15645 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:45307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab
--principal=kudu/127.7.43.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.43.66
--webserver_port=0
--tserver_master_addrs=127.7.43.126:38483
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15645
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.66
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:27.531528 15645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:27.531693 15645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:27.536352 15653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:27.536489 15656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:27.536399 15645 server_base.cc:1034] running on GCE node
W20260806 06:31:27.536504 15654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:27.536849 15645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:27.537060 15645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:27.538204 15645 hybrid_clock.cc:648] HybridClock initialized: now 1785997887538192 us; error 28 us; skew 500 ppm
Aug 06 06:31:27 dist-test-slave-6zbq krb5kdc[15299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997887, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:27.540657 15645 init.cc:377] Logged in from keytab as kudu/127.7.43.66@KRBTEST.COM (short username kudu)
I20260806 06:31:27.541008 15645 webserver.cc:460] Webserver started at http://127.7.43.66:45527/ using document root <none> and password file <none>
I20260806 06:31:27.541199 15645 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:27.541237 15645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:27.541321 15645 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:27.542207 15645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/ts-1/data/instance:
uuid: "afdc37b483b744ec9187793d7ba58eb1"
format_stamp: "Formatted at 2026-08-06 06:31:27 on dist-test-slave-6zbq"
server_key: "a1d853acddb8b899a49fdc1b87a24c78"
server_key_iv: "912b13ce34ff0f5b801f1664eef9c3b3"
server_key_version: "encryptionkey@0"
I20260806 06:31:27.542541 15645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/ts-1/wal/instance:
uuid: "afdc37b483b744ec9187793d7ba58eb1"
format_stamp: "Formatted at 2026-08-06 06:31:27 on dist-test-slave-6zbq"
server_key: "a1d853acddb8b899a49fdc1b87a24c78"
server_key_iv: "912b13ce34ff0f5b801f1664eef9c3b3"
server_key_version: "encryptionkey@0"
I20260806 06:31:27.543655 15645 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:31:27.544247 15665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:27.544417 15645 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:31:27.544503 15645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/ts-1/wal
uuid: "afdc37b483b744ec9187793d7ba58eb1"
format_stamp: "Formatted at 2026-08-06 06:31:27 on dist-test-slave-6zbq"
server_key: "a1d853acddb8b899a49fdc1b87a24c78"
server_key_iv: "912b13ce34ff0f5b801f1664eef9c3b3"
server_key_version: "encryptionkey@0"
I20260806 06:31:27.544569 15645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:27.563953 15645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:27.565600 15645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:27.565723 15645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:27.565905 15645 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:31:27.566179 15645 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:31:27.566212 15645 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:27.566239 15645 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:31:27.566265 15645 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:27.571522 15645 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.66:44833
I20260806 06:31:27.571560 15840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.66:44833 every 8 connection(s)
I20260806 06:31:27.571826 15645 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
I20260806 06:31:27.574862 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 15645
I20260806 06:31:27.574978 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/ts-1/wal/instance
I20260806 06:31:27.575173 7341 external_mini_cluster.cc:1428] Setting key 8bf27986f79292b38eb5f631ad886652
Aug 06 06:31:27 dist-test-slave-6zbq krb5kdc[15299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997887, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.66@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:31:27.580629 15423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:27.572476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:59225 (local address 127.7.43.126:38483)
0806 06:31:27.572539 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:27.572541 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:27.572693 (+ 152us) server_negotiation.cc:407] Connection header received
0806 06:31:27.573543 (+ 850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:27.573544 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:27.573553 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:27.573588 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:27.574115 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:27.574633 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:27.575893 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:27.576083 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:27.576128 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:27.578175 (+ 2047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:27.578176 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:27.578177 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:27.578184 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:27.579454 (+ 1270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:27.579804 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:27.579805 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:27.579806 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:27.579856 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:27.580058 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:27.580059 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:27.580059 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:27.580159 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:27.580214 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:27.580459 (+ 245us) server_negotiation.cc:299] Negotiation successful
0806 06:31:27.580542 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 06:31:27.580760 15845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:27.572433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38483 (local address 127.7.43.66:59225)
0806 06:31:27.572582 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:27.572620 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:27.573450 (+ 830us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:27.573649 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:27.573652 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:27.573880 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:27.574050 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:27.574055 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:27.574707 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:27.574708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:27.575809 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:27.575812 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:27.576155 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:27.576156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:27.576281 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:27.576688 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:27.576698 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:27.578092 (+ 1394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:27.579526 (+ 1434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:27.579533 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:27.579535 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:27.579738 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:27.579909 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:27.579910 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:27.579910 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:27.580003 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:27.580212 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:27.580219 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:27.580345 (+ 126us) client_negotiation.cc:769] Sending connection context
0806 06:31:27.580428 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 06:31:27.580568 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":79,"threads_started":1}
I20260806 06:31:27.581158 15841 heartbeater.cc:347] Connected to a master server at 127.7.43.126:38483
I20260806 06:31:27.581239 15841 heartbeater.cc:464] Registering TS with master...
I20260806 06:31:27.581393 15841 heartbeater.cc:511] Master 127.7.43.126:38483 requested a full tablet report, sending...
I20260806 06:31:27.581727 15365 ts_manager.cc:194] Registered new tserver with Master: afdc37b483b744ec9187793d7ba58eb1 (127.7.43.66:44833)
I20260806 06:31:27.582073 15365 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.66@KRBTEST.COM'} at 127.7.43.66:59225
WARNING: no policy specified for kudu/127.7.43.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab.
Entry for principal kudu/127.7.43.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab.
I20260806 06:31:27.597628 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.67:0
--local_ip_for_outbound_sockets=127.7.43.67
--webserver_interface=127.7.43.67
--webserver_port=0
--tserver_master_addrs=127.7.43.126:38483
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab
--principal=kudu/127.7.43.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:45307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:31:27.676574 15850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:27.676752 15850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:27.676774 15850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:27.677898 15850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:27.677942 15850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:27.677965 15850 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.67
I20260806 06:31:27.679441 15850 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:45307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab
--principal=kudu/127.7.43.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.43.67
--webserver_port=0
--tserver_master_addrs=127.7.43.126:38483
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15850
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.67
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:27.679648 15850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:27.679843 15850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:27.684371 15861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:27.684515 15850 server_base.cc:1034] running on GCE node
W20260806 06:31:27.684423 15857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:27.684384 15858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:27.684842 15850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:27.685041 15850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:27.686167 15850 hybrid_clock.cc:648] HybridClock initialized: now 1785997887686152 us; error 32 us; skew 500 ppm
Aug 06 06:31:27 dist-test-slave-6zbq krb5kdc[15299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997887, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:27.688490 15850 init.cc:377] Logged in from keytab as kudu/127.7.43.67@KRBTEST.COM (short username kudu)
I20260806 06:31:27.688866 15850 webserver.cc:460] Webserver started at http://127.7.43.67:40263/ using document root <none> and password file <none>
I20260806 06:31:27.689055 15850 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:27.689097 15850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:27.689217 15850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:27.690141 15850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/ts-2/data/instance:
uuid: "08d404acf48b4f9a8a2e37557fde4713"
format_stamp: "Formatted at 2026-08-06 06:31:27 on dist-test-slave-6zbq"
server_key: "e1a30d34eafce60403d692637df1dad1"
server_key_iv: "a0469cd5d09ef97e23c6d6548efd316c"
server_key_version: "encryptionkey@0"
I20260806 06:31:27.690498 15850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/ts-2/wal/instance:
uuid: "08d404acf48b4f9a8a2e37557fde4713"
format_stamp: "Formatted at 2026-08-06 06:31:27 on dist-test-slave-6zbq"
server_key: "e1a30d34eafce60403d692637df1dad1"
server_key_iv: "a0469cd5d09ef97e23c6d6548efd316c"
server_key_version: "encryptionkey@0"
I20260806 06:31:27.691689 15850 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:31:27.692332 15872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:27.692492 15850 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:31:27.692572 15850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/ts-2/wal
uuid: "08d404acf48b4f9a8a2e37557fde4713"
format_stamp: "Formatted at 2026-08-06 06:31:27 on dist-test-slave-6zbq"
server_key: "e1a30d34eafce60403d692637df1dad1"
server_key_iv: "a0469cd5d09ef97e23c6d6548efd316c"
server_key_version: "encryptionkey@0"
I20260806 06:31:27.692631 15850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:27.699734 15850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:27.701381 15850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:27.701495 15850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:27.701666 15850 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:31:27.701896 15850 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:31:27.701931 15850 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:27.701951 15850 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:31:27.701967 15850 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:27.707300 15850 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.67:46865
I20260806 06:31:27.707368 16045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.67:46865 every 8 connection(s)
I20260806 06:31:27.707619 15850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
I20260806 06:31:27.711643 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 15850
I20260806 06:31:27.711731 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/minicluster-data/ts-2/wal/instance
I20260806 06:31:27.711905 7341 external_mini_cluster.cc:1428] Setting key cb89271ec0d6cc2e29fcb84957dbf0fb
Aug 06 06:31:27 dist-test-slave-6zbq krb5kdc[15299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997887, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.67@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:31:27.715867 15423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:27.708253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:38479 (local address 127.7.43.126:38483)
0806 06:31:27.708323 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:27.708325 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:27.708457 (+ 132us) server_negotiation.cc:407] Connection header received
0806 06:31:27.709292 (+ 835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:27.709293 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:27.709304 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:27.709344 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:27.709864 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:27.710333 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:27.711356 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:27.711574 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:27.711650 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:27.713722 (+ 2072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:27.713723 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:27.713723 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:27.713730 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:27.714844 (+ 1114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:27.715198 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:27.715199 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:27.715199 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:27.715252 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:27.715431 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:27.715431 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:27.715431 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:27.715524 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:27.715567 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:27.715750 (+ 183us) server_negotiation.cc:299] Negotiation successful
0806 06:31:27.715807 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 06:31:27.716022 16052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:27.708211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38483 (local address 127.7.43.67:38479)
0806 06:31:27.708355 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:27.708387 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:27.709196 (+ 809us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:27.709395 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:27.709398 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:27.709631 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:27.709800 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:27.709804 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:27.710434 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:27.710434 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:27.711275 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:27.711277 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:27.711651 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:27.711651 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:27.711759 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:27.712224 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:27.712234 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:27.713635 (+ 1401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:27.714924 (+ 1289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:27.714930 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:27.714931 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:27.715134 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:27.715302 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:27.715302 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:27.715302 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:27.715388 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:27.715563 (+ 175us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:27.715566 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:27.715661 (+ 95us) client_negotiation.cc:769] Sending connection context
0806 06:31:27.715731 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 06:31:27.715848 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":70,"threads_started":1}
I20260806 06:31:27.716420 16046 heartbeater.cc:347] Connected to a master server at 127.7.43.126:38483
I20260806 06:31:27.716501 16046 heartbeater.cc:464] Registering TS with master...
I20260806 06:31:27.716658 16046 heartbeater.cc:511] Master 127.7.43.126:38483 requested a full tablet report, sending...
I20260806 06:31:27.716989 15365 ts_manager.cc:194] Registered new tserver with Master: 08d404acf48b4f9a8a2e37557fde4713 (127.7.43.67:46865)
I20260806 06:31:27.717329 15365 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.67@KRBTEST.COM'} at 127.7.43.67:38479
I20260806 06:31:27.723351 7341 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/krb5kdc/test-admin.keytab.
Aug 06 06:31:27 dist-test-slave-6zbq krb5kdc[15299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:27.736480 7341 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260806 06:31:27.737844 16057 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:27.737247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50332 (local address 127.7.43.65:33283)
0806 06:31:27.737385 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:27.737389 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:27.737397 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:31:27.737419 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:27.737421 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:27.737433 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:27.737492 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:27.737768 (+ 276us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50332: BlockingRecv error: recv got EOF from 127.0.0.1:50332 (error 108)
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":62,"threads_started":1}
W20260806 06:31:27.737913 7341 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/krb5kdc/krb5cc)
Aug 06 06:31:27 dist-test-slave-6zbq krb5kdc[15299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:27.738804 7341 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 06:31:27 dist-test-slave-6zbq krb5kdc[15299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:27.739573 7341 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 06:31:27 dist-test-slave-6zbq krb5kdc[15299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.43.65@KRBTEST.COM
I20260806 06:31:27.746079 16057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:27.740018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50340 (local address 127.7.43.65:33283)
0806 06:31:27.740070 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:27.740072 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:27.740100 (+ 28us) server_negotiation.cc:407] Connection header received
0806 06:31:27.740155 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:27.740155 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:27.740165 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:27.740205 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:27.740531 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:27.741213 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:27.741875 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:27.742168 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:27.742222 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:27.744018 (+ 1796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:27.744026 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:27.744026 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:27.744032 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:27.745227 (+ 1195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:27.745517 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:27.745519 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:27.745520 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:27.745574 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:27.745696 (+ 122us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:27.745696 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:27.745696 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:27.745800 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:27.745895 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:27.745930 (+ 35us) server_negotiation.cc:299] Negotiation successful
0806 06:31:27.745985 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
W20260806 06:31:27.747675 16057 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:27.747203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50348 (local address 127.7.43.65:33283)
0806 06:31:27.747240 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:27.747242 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:27.747276 (+ 34us) server_negotiation.cc:407] Connection header received
0806 06:31:27.747330 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:27.747331 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:27.747352 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:27.747395 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:27.747636 (+ 241us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50348: BlockingRecv error: recv got EOF from 127.0.0.1:50348 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260806 06:31:27.747735 7341 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/krb5kdc/krb5cc' of type FILE
Aug 06 06:31:27 dist-test-slave-6zbq krb5kdc[15299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:27.748579 7341 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 06:31:27 dist-test-slave-6zbq krb5kdc[15299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:27.749485 7341 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 06:31:27 dist-test-slave-6zbq krb5kdc[15299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.43.65@KRBTEST.COM
I20260806 06:31:27.759371 16057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:27.750003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50364 (local address 127.7.43.65:33283)
0806 06:31:27.750038 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:27.750040 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:27.750381 (+ 341us) server_negotiation.cc:407] Connection header received
0806 06:31:27.750457 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:27.750458 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:27.750468 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:27.750512 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:27.750981 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:27.751726 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:27.752676 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:27.752947 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:27.753011 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:27.755326 (+ 2315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:27.755327 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:27.755327 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:27.755334 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:27.758198 (+ 2864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:27.758644 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:27.758645 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:27.758645 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:27.758702 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:27.758896 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:27.758896 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:27.758896 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:27.758979 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:27.759041 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:27.759208 (+ 167us) server_negotiation.cc:299] Negotiation successful
0806 06:31:27.759296 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260806 06:31:27.760895 16057 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:27.760384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50372 (local address 127.7.43.65:33283)
0806 06:31:27.760420 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:27.760422 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:27.760476 (+ 54us) server_negotiation.cc:407] Connection header received
0806 06:31:27.760532 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:27.760533 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:27.760542 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:27.760595 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:27.760856 (+ 261us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50372: BlockingRecv error: recv got EOF from 127.0.0.1:50372 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260806 06:31:27.760910 7341 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785997861828271-7341-0/krb5kdc/krb5cc)
Aug 06 06:31:27 dist-test-slave-6zbq krb5kdc[15299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:27.761865 7341 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 06:31:27 dist-test-slave-6zbq krb5kdc[15299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:27.762967 7341 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 06:31:27 dist-test-slave-6zbq krb5kdc[15299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.43.65@KRBTEST.COM
I20260806 06:31:27.772673 16057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:27.763479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50386 (local address 127.7.43.65:33283)
0806 06:31:27.763520 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:27.763522 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:27.763553 (+ 31us) server_negotiation.cc:407] Connection header received
0806 06:31:27.763617 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:27.763618 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:27.763625 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:27.763668 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:27.763986 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:27.764430 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:27.765175 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:27.765372 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:27.765399 (+ 27us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:27.770318 (+ 4919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:27.770320 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:27.770320 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:27.770364 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:27.771721 (+ 1357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:27.772088 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:27.772088 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:27.772088 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:27.772134 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:27.772311 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:27.772311 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:27.772311 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:27.772385 (+ 74us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:27.772446 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:27.772571 (+ 125us) server_negotiation.cc:299] Negotiation successful
0806 06:31:27.772619 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 06:31:27.773245 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 15449
I20260806 06:31:27.789588 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 15645
I20260806 06:31:27.807672 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 15850
I20260806 06:31:27.814291 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 15317
2026-08-06T06:31:27Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1888 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:31:27 dist-test-slave-6zbq krb5kdc[16073](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:31:27 dist-test-slave-6zbq krb5kdc[16073](info): set up 2 sockets
Aug 06 06:31:27 dist-test-slave-6zbq krb5kdc[16073](info): commencing operation
krb5kdc: starting...
W20260806 06:31:28.914096 7341 mini_kdc.cc:121] Time spent starting KDC: real 1.079s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Aug 06 06:31:28 dist-test-slave-6zbq krb5kdc[16073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997888, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:31:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:31:28Z Disabled control of system clock
WARNING: no policy specified for oryx/127.7.43.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.43.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.43.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/oryx_127.7.43.126.keytab.
Entry for principal oryx/127.7.43.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/oryx_127.7.43.126.keytab.
I20260806 06:31:29.018563 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:36787
--webserver_interface=127.7.43.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/oryx_127.7.43.126.keytab
--principal=oryx/127.7.43.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:37673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.43.126:36787
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:31:29.095829 16094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:29.095997 16094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:29.096017 16094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:29.097108 16094 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:31:29.097149 16094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:29.097162 16094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:29.097173 16094 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:31:29.097184 16094 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:31:29.098637 16094 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:37673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.43.126:36787
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/oryx_127.7.43.126.keytab
--principal=oryx/127.7.43.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:36787
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.43.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16094
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:29.098850 16094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:29.099046 16094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:29.103407 16106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:29.103408 16100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:29.103471 16099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:29.103540 16094 server_base.cc:1034] running on GCE node
I20260806 06:31:29.103842 16094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:29.104101 16094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:29.105232 16094 hybrid_clock.cc:648] HybridClock initialized: now 1785997889105219 us; error 31 us; skew 500 ppm
Aug 06 06:31:29 dist-test-slave-6zbq krb5kdc[16073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997889, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.43.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:29.107434 16094 init.cc:377] Logged in from keytab as oryx/127.7.43.126@KRBTEST.COM (short username oryx)
I20260806 06:31:29.107797 16094 webserver.cc:460] Webserver started at http://127.7.43.126:38079/ using document root <none> and password file <none>
I20260806 06:31:29.107995 16094 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:29.108040 16094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:29.108155 16094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:29.109062 16094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-0/data/instance:
uuid: "ee7ac2b4afd849459e289fe5dba19836"
format_stamp: "Formatted at 2026-08-06 06:31:29 on dist-test-slave-6zbq"
server_key: "fa06d1a0cf7b560e1cd9ec923398404e"
server_key_iv: "c5eda7308462c75963d86b4ba8c22bf8"
server_key_version: "encryptionkey@0"
I20260806 06:31:29.109369 16094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-0/wal/instance:
uuid: "ee7ac2b4afd849459e289fe5dba19836"
format_stamp: "Formatted at 2026-08-06 06:31:29 on dist-test-slave-6zbq"
server_key: "fa06d1a0cf7b560e1cd9ec923398404e"
server_key_iv: "c5eda7308462c75963d86b4ba8c22bf8"
server_key_version: "encryptionkey@0"
I20260806 06:31:29.110536 16094 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:31:29.111130 16114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:29.111289 16094 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:29.111379 16094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "ee7ac2b4afd849459e289fe5dba19836"
format_stamp: "Formatted at 2026-08-06 06:31:29 on dist-test-slave-6zbq"
server_key: "fa06d1a0cf7b560e1cd9ec923398404e"
server_key_iv: "c5eda7308462c75963d86b4ba8c22bf8"
server_key_version: "encryptionkey@0"
I20260806 06:31:29.111439 16094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:29.117451 16094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:29.118994 16094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:29.119120 16094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:29.122655 16094 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.126:36787
I20260806 06:31:29.122682 16196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.126:36787 every 8 connection(s)
I20260806 06:31:29.122961 16094 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:31:29.123826 16197 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:29.125793 16197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836: Bootstrap starting.
I20260806 06:31:29.126312 16197 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:29.126587 16197 log.cc:824] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836: Log is configured to *not* fsync() on all Append() calls
I20260806 06:31:29.127187 16197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836: No bootstrap required, opened a new log
I20260806 06:31:29.128671 16197 raft_consensus.cc:348] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } }
I20260806 06:31:29.128795 16197 raft_consensus.cc:374] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:29.128827 16197 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee7ac2b4afd849459e289fe5dba19836, State: Initialized, Role: FOLLOWER
I20260806 06:31:29.128969 16197 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } }
I20260806 06:31:29.129043 16197 raft_consensus.cc:388] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:31:29.129078 16197 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:31:29.129123 16197 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:29.129848 16197 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } }
I20260806 06:31:29.129938 16197 leader_election.cc:302] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ee7ac2b4afd849459e289fe5dba19836; no voters:
I20260806 06:31:29.130079 16197 leader_election.cc:288] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:31:29.130115 16204 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:31:29.130250 16204 raft_consensus.cc:686] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836 [term 1 LEADER]: Becoming Leader. State: Replica: ee7ac2b4afd849459e289fe5dba19836, State: Running, Role: LEADER
I20260806 06:31:29.130285 16197 sys_catalog.cc:564] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:31:29.130358 16204 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } }
I20260806 06:31:29.130714 16207 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ee7ac2b4afd849459e289fe5dba19836" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } } }
I20260806 06:31:29.130792 16207 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836 [sys.catalog]: This master's current role is: LEADER
I20260806 06:31:29.130870 16208 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee7ac2b4afd849459e289fe5dba19836. Latest consensus state: current_term: 1 leader_uuid: "ee7ac2b4afd849459e289fe5dba19836" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } } }
I20260806 06:31:29.131073 16220 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:31:29.131091 16208 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836 [sys.catalog]: This master's current role is: LEADER
I20260806 06:31:29.131541 16220 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:31:29.132324 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 16094
I20260806 06:31:29.132427 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-0/wal/instance
I20260806 06:31:29.132596 7341 external_mini_cluster.cc:1428] Setting key d02cfb8ae5517c2436f3c6b819b26a64
I20260806 06:31:29.132987 16220 catalog_manager.cc:1347] Generated new cluster ID: f88ac3451d944ee6994511cbff6c6a75
I20260806 06:31:29.133036 16220 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:31:29 dist-test-slave-6zbq krb5kdc[16073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997888, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM
I20260806 06:31:29.142444 16234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:29.133176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45610 (local address 127.7.43.126:36787)
0806 06:31:29.133315 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:29.133323 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:29.134523 (+ 1200us) server_negotiation.cc:407] Connection header received
0806 06:31:29.134655 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:29.134660 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:29.134699 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:29.134802 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:29.135423 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:29.136305 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:29.137014 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:29.137393 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:29.137492 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:29.139571 (+ 2079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:29.139575 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:29.139575 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:29.139585 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:29.141024 (+ 1439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:29.141355 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:29.141357 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:29.141358 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:29.141447 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:29.141621 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:29.141622 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:29.141623 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:29.141846 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:29.141979 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:29.142086 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 06:31:29.142214 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":64,"threads_started":1}
I20260806 06:31:29.143088 16220 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:31:29.143745 16220 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:31:29.152980 16220 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836: Generated new TSK 0
I20260806 06:31:29.153117 16220 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.7.43.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.43.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.43.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/oryx_127.7.43.65.keytab.
Entry for principal oryx/127.7.43.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/oryx_127.7.43.65.keytab.
I20260806 06:31:29.185043 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.65:0
--local_ip_for_outbound_sockets=127.7.43.65
--webserver_interface=127.7.43.65
--webserver_port=0
--tserver_master_addrs=127.7.43.126:36787
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/oryx_127.7.43.65.keytab
--principal=oryx/127.7.43.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:37673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:31:29.269797 16238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:29.269980 16238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:29.270011 16238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:29.271148 16238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:29.271191 16238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:29.271227 16238 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.65
I20260806 06:31:29.272634 16238 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:37673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/oryx_127.7.43.65.keytab
--principal=oryx/127.7.43.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.43.65
--webserver_port=0
--tserver_master_addrs=127.7.43.126:36787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16238
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.65
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:29.272853 16238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:29.273053 16238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:29.277522 16249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:29.277537 16246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:29.277601 16245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:29.277724 16238 server_base.cc:1034] running on GCE node
I20260806 06:31:29.277895 16238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:29.278103 16238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:29.279233 16238 hybrid_clock.cc:648] HybridClock initialized: now 1785997889279223 us; error 25 us; skew 500 ppm
Aug 06 06:31:29 dist-test-slave-6zbq krb5kdc[16073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997889, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.43.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:29.281528 16238 init.cc:377] Logged in from keytab as oryx/127.7.43.65@KRBTEST.COM (short username oryx)
I20260806 06:31:29.281903 16238 webserver.cc:460] Webserver started at http://127.7.43.65:45349/ using document root <none> and password file <none>
I20260806 06:31:29.282105 16238 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:29.282150 16238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:29.282274 16238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:29.283219 16238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-0/data/instance:
uuid: "17fc103e74544c2a98d06b27aabb9435"
format_stamp: "Formatted at 2026-08-06 06:31:29 on dist-test-slave-6zbq"
server_key: "9d6717dbb8f470ce6b7d0f2361149673"
server_key_iv: "b55d4c3090124ce71c5a80b159a7f2b9"
server_key_version: "encryptionkey@0"
I20260806 06:31:29.283541 16238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-0/wal/instance:
uuid: "17fc103e74544c2a98d06b27aabb9435"
format_stamp: "Formatted at 2026-08-06 06:31:29 on dist-test-slave-6zbq"
server_key: "9d6717dbb8f470ce6b7d0f2361149673"
server_key_iv: "b55d4c3090124ce71c5a80b159a7f2b9"
server_key_version: "encryptionkey@0"
I20260806 06:31:29.284704 16238 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:31:29.285324 16259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:29.285472 16238 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:31:29.285554 16238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-0/wal
uuid: "17fc103e74544c2a98d06b27aabb9435"
format_stamp: "Formatted at 2026-08-06 06:31:29 on dist-test-slave-6zbq"
server_key: "9d6717dbb8f470ce6b7d0f2361149673"
server_key_iv: "b55d4c3090124ce71c5a80b159a7f2b9"
server_key_version: "encryptionkey@0"
I20260806 06:31:29.285614 16238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:29.298018 16238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:29.300025 16238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:29.300151 16238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:29.300348 16238 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:31:29.300618 16238 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:31:29.300652 16238 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:29.300678 16238 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:31:29.300702 16238 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:29.306164 16238 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.65:39903
I20260806 06:31:29.306200 16426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.65:39903 every 8 connection(s)
I20260806 06:31:29.306510 16238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
I20260806 06:31:29.309696 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 16238
I20260806 06:31:29.309793 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-0/wal/instance
I20260806 06:31:29.309973 7341 external_mini_cluster.cc:1428] Setting key b74d3df192de5ae4415725094b3ebc59
Aug 06 06:31:29 dist-test-slave-6zbq krb5kdc[16073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997889, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.43.65@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM
I20260806 06:31:29.315280 16234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:29.307119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:44987 (local address 127.7.43.126:36787)
0806 06:31:29.307167 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:29.307170 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:29.307339 (+ 169us) server_negotiation.cc:407] Connection header received
0806 06:31:29.307784 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:29.307785 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:29.307797 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:29.307857 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:29.308688 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:29.309175 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:29.310365 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:29.310574 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:29.310632 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:29.312687 (+ 2055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:29.312689 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:29.312690 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:29.312698 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:29.313964 (+ 1266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:29.314369 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:29.314370 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:29.314370 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:29.314423 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:29.314660 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:29.314661 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:29.314661 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:29.314780 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:29.314837 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:29.315100 (+ 263us) server_negotiation.cc:299] Negotiation successful
0806 06:31:29.315193 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:31:29.315413 16432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:29.307096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:36787 (local address 127.7.43.65:44987)
0806 06:31:29.307243 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:29.307276 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:29.307709 (+ 433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:29.307913 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:29.307915 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:29.308435 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:29.308611 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:29.308616 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:29.309234 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:29.309235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:29.310259 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:29.310262 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:29.310648 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:29.310650 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:29.310776 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:29.311166 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:29.311174 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:29.312598 (+ 1424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:29.314031 (+ 1433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:29.314039 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:29.314041 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:29.314280 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:29.314478 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:29.314479 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:29.314479 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:29.314579 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:29.314837 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:29.314844 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:29.314967 (+ 123us) client_negotiation.cc:769] Sending connection context
0806 06:31:29.315076 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 06:31:29.315220 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":84,"threads_started":1}
I20260806 06:31:29.315845 16427 heartbeater.cc:347] Connected to a master server at 127.7.43.126:36787
I20260806 06:31:29.315923 16427 heartbeater.cc:464] Registering TS with master...
I20260806 06:31:29.316074 16427 heartbeater.cc:511] Master 127.7.43.126:36787 requested a full tablet report, sending...
I20260806 06:31:29.316572 16142 ts_manager.cc:194] Registered new tserver with Master: 17fc103e74544c2a98d06b27aabb9435 (127.7.43.65:39903)
I20260806 06:31:29.317150 16142 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.43.65@KRBTEST.COM'} at 127.7.43.65:44987
WARNING: no policy specified for oryx/127.7.43.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.43.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.43.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/oryx_127.7.43.66.keytab.
Entry for principal oryx/127.7.43.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/oryx_127.7.43.66.keytab.
I20260806 06:31:29.332381 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.66:0
--local_ip_for_outbound_sockets=127.7.43.66
--webserver_interface=127.7.43.66
--webserver_port=0
--tserver_master_addrs=127.7.43.126:36787
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/oryx_127.7.43.66.keytab
--principal=oryx/127.7.43.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:37673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:31:29.412732 16436 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:29.412935 16436 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:29.412958 16436 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:29.414103 16436 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:29.414145 16436 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:29.414172 16436 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.66
I20260806 06:31:29.415583 16436 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:37673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/oryx_127.7.43.66.keytab
--principal=oryx/127.7.43.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.43.66
--webserver_port=0
--tserver_master_addrs=127.7.43.126:36787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16436
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.66
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:29.415751 16436 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:29.415925 16436 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:29.420403 16448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:29.420403 16444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:29.420481 16443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:29.420548 16436 server_base.cc:1034] running on GCE node
I20260806 06:31:29.420715 16436 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:29.420926 16436 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:29.422060 16436 hybrid_clock.cc:648] HybridClock initialized: now 1785997889422046 us; error 28 us; skew 500 ppm
Aug 06 06:31:29 dist-test-slave-6zbq krb5kdc[16073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997889, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.43.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:29.424405 16436 init.cc:377] Logged in from keytab as oryx/127.7.43.66@KRBTEST.COM (short username oryx)
I20260806 06:31:29.424751 16436 webserver.cc:460] Webserver started at http://127.7.43.66:34127/ using document root <none> and password file <none>
I20260806 06:31:29.424911 16436 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:29.424947 16436 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:29.425031 16436 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:29.425915 16436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-1/data/instance:
uuid: "40b11f4fbad9437eb9f11d2da5dd0d8b"
format_stamp: "Formatted at 2026-08-06 06:31:29 on dist-test-slave-6zbq"
server_key: "5606a74422eec41a4a26837d7cb30bb5"
server_key_iv: "c7f21048e61f909184dbcda8f95756ad"
server_key_version: "encryptionkey@0"
I20260806 06:31:29.426187 16436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-1/wal/instance:
uuid: "40b11f4fbad9437eb9f11d2da5dd0d8b"
format_stamp: "Formatted at 2026-08-06 06:31:29 on dist-test-slave-6zbq"
server_key: "5606a74422eec41a4a26837d7cb30bb5"
server_key_iv: "c7f21048e61f909184dbcda8f95756ad"
server_key_version: "encryptionkey@0"
I20260806 06:31:29.427394 16436 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:31:29.428048 16458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:29.428205 16436 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:31:29.428290 16436 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-1/wal
uuid: "40b11f4fbad9437eb9f11d2da5dd0d8b"
format_stamp: "Formatted at 2026-08-06 06:31:29 on dist-test-slave-6zbq"
server_key: "5606a74422eec41a4a26837d7cb30bb5"
server_key_iv: "c7f21048e61f909184dbcda8f95756ad"
server_key_version: "encryptionkey@0"
I20260806 06:31:29.428344 16436 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:29.444759 16436 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:29.446249 16436 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:29.446389 16436 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:29.446560 16436 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:31:29.446776 16436 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:31:29.446803 16436 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:29.446828 16436 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:31:29.446843 16436 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:29.451985 16436 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.66:44525
I20260806 06:31:29.452049 16627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.66:44525 every 8 connection(s)
I20260806 06:31:29.452282 16436 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
I20260806 06:31:29.456432 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 16436
I20260806 06:31:29.456530 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-1/wal/instance
I20260806 06:31:29.456707 7341 external_mini_cluster.cc:1428] Setting key 7c2c8d6e08c4ee30600ca9575699219f
Aug 06 06:31:29 dist-test-slave-6zbq krb5kdc[16073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997889, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.43.66@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM
I20260806 06:31:29.460489 16234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:29.452842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:44881 (local address 127.7.43.126:36787)
0806 06:31:29.452896 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:29.452898 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:29.453085 (+ 187us) server_negotiation.cc:407] Connection header received
0806 06:31:29.453507 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:29.453508 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:29.453519 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:29.453565 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:29.454402 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:29.454883 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:29.455755 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:29.455936 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:29.455978 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:29.458003 (+ 2025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:29.458004 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:29.458004 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:29.458010 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:29.459265 (+ 1255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:29.459625 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:29.459626 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:29.459626 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:29.459686 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:29.459883 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:29.459884 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:29.459884 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:29.460013 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:29.460081 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:29.460298 (+ 217us) server_negotiation.cc:299] Negotiation successful
0806 06:31:29.460394 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 06:31:29.460587 16631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:29.452809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:36787 (local address 127.7.43.66:44881)
0806 06:31:29.452959 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:29.452993 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:29.453428 (+ 435us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:29.453612 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:29.453614 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:29.454121 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:29.454292 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:29.454296 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:29.454946 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:29.454947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:29.455691 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:29.455693 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:29.455987 (+ 294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:29.455987 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:29.456073 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:29.456476 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:29.456483 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:29.457916 (+ 1433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:29.459354 (+ 1438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:29.459360 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:29.459362 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:29.459564 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:29.459749 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:29.459749 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:29.459750 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:29.459829 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:29.460071 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:29.460074 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:29.460169 (+ 95us) client_negotiation.cc:769] Sending connection context
0806 06:31:29.460248 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 06:31:29.460396 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":80,"threads_started":1}
I20260806 06:31:29.460953 16628 heartbeater.cc:347] Connected to a master server at 127.7.43.126:36787
I20260806 06:31:29.461033 16628 heartbeater.cc:464] Registering TS with master...
I20260806 06:31:29.461196 16628 heartbeater.cc:511] Master 127.7.43.126:36787 requested a full tablet report, sending...
I20260806 06:31:29.461534 16142 ts_manager.cc:194] Registered new tserver with Master: 40b11f4fbad9437eb9f11d2da5dd0d8b (127.7.43.66:44525)
I20260806 06:31:29.461882 16142 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.43.66@KRBTEST.COM'} at 127.7.43.66:44881
WARNING: no policy specified for oryx/127.7.43.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.43.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.43.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/oryx_127.7.43.67.keytab.
Entry for principal oryx/127.7.43.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/oryx_127.7.43.67.keytab.
I20260806 06:31:29.478788 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.67:0
--local_ip_for_outbound_sockets=127.7.43.67
--webserver_interface=127.7.43.67
--webserver_port=0
--tserver_master_addrs=127.7.43.126:36787
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/oryx_127.7.43.67.keytab
--principal=oryx/127.7.43.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:37673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:31:29.555737 16634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:29.555912 16634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:29.555943 16634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:29.557010 16634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:29.557046 16634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:29.557081 16634 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.67
I20260806 06:31:29.558498 16634 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:37673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/oryx_127.7.43.67.keytab
--principal=oryx/127.7.43.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.43.67
--webserver_port=0
--tserver_master_addrs=127.7.43.126:36787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16634
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.67
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:29.558703 16634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:29.558895 16634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:29.563349 16645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:29.563366 16648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:29.563413 16643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:29.563496 16634 server_base.cc:1034] running on GCE node
I20260806 06:31:29.563741 16634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:29.563970 16634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:29.565097 16634 hybrid_clock.cc:648] HybridClock initialized: now 1785997889565079 us; error 32 us; skew 500 ppm
Aug 06 06:31:29 dist-test-slave-6zbq krb5kdc[16073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997889, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.43.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:29.567695 16634 init.cc:377] Logged in from keytab as oryx/127.7.43.67@KRBTEST.COM (short username oryx)
I20260806 06:31:29.568141 16634 webserver.cc:460] Webserver started at http://127.7.43.67:32991/ using document root <none> and password file <none>
I20260806 06:31:29.568326 16634 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:29.568373 16634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:29.568496 16634 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:29.569412 16634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-2/data/instance:
uuid: "930504b65be34a37ac7455cda9c9c7b6"
format_stamp: "Formatted at 2026-08-06 06:31:29 on dist-test-slave-6zbq"
server_key: "c3e89baab1c20d8e30d864100246eb6a"
server_key_iv: "b0a6e6ffbb682bd7834eec1bdcd28f84"
server_key_version: "encryptionkey@0"
I20260806 06:31:29.569720 16634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-2/wal/instance:
uuid: "930504b65be34a37ac7455cda9c9c7b6"
format_stamp: "Formatted at 2026-08-06 06:31:29 on dist-test-slave-6zbq"
server_key: "c3e89baab1c20d8e30d864100246eb6a"
server_key_iv: "b0a6e6ffbb682bd7834eec1bdcd28f84"
server_key_version: "encryptionkey@0"
I20260806 06:31:29.570947 16634 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:31:29.571543 16661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:29.571686 16634 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:31:29.571766 16634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-2/wal
uuid: "930504b65be34a37ac7455cda9c9c7b6"
format_stamp: "Formatted at 2026-08-06 06:31:29 on dist-test-slave-6zbq"
server_key: "c3e89baab1c20d8e30d864100246eb6a"
server_key_iv: "b0a6e6ffbb682bd7834eec1bdcd28f84"
server_key_version: "encryptionkey@0"
I20260806 06:31:29.571823 16634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:29.583071 16634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:29.584589 16634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:29.584708 16634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:29.584887 16634 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:31:29.585141 16634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:31:29.585173 16634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:29.585199 16634 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:31:29.585224 16634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:29.590709 16634 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.67:41259
I20260806 06:31:29.590773 16838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.67:41259 every 8 connection(s)
I20260806 06:31:29.591023 16634 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
I20260806 06:31:29.592607 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 16634
I20260806 06:31:29.592682 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-2/wal/instance
I20260806 06:31:29.592854 7341 external_mini_cluster.cc:1428] Setting key e9c2b1809be827a41af24e3a286cc140
Aug 06 06:31:29 dist-test-slave-6zbq krb5kdc[16073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997889, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.43.67@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM
I20260806 06:31:29.599138 16234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:29.591614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:39579 (local address 127.7.43.126:36787)
0806 06:31:29.591671 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:29.591673 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:29.591843 (+ 170us) server_negotiation.cc:407] Connection header received
0806 06:31:29.592278 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:29.592279 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:29.592289 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:29.592324 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:29.593320 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:29.593811 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:29.594810 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:29.595012 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:29.595067 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:29.596846 (+ 1779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:29.596848 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:29.596848 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:29.596856 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:29.597884 (+ 1028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:29.598236 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:29.598237 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:29.598237 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:29.598292 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:29.598534 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:29.598535 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:29.598535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:29.598642 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:29.598700 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:29.598948 (+ 248us) server_negotiation.cc:299] Negotiation successful
0806 06:31:29.599049 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 06:31:29.599251 16844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:29.591582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:36787 (local address 127.7.43.67:39579)
0806 06:31:29.591729 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:29.591766 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:29.592190 (+ 424us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:29.592383 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:29.592386 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:29.592993 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:29.593245 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:29.593252 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:29.593900 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:29.593901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:29.594732 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:29.594735 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:29.595079 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:29.595079 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:29.595202 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:29.595555 (+ 353us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:29.595562 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:29.596754 (+ 1192us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:29.597956 (+ 1202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:29.597963 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:29.597965 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:29.598168 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:29.598374 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:29.598375 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:29.598375 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:29.598476 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:29.598698 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:29.598705 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:29.598829 (+ 124us) client_negotiation.cc:769] Sending connection context
0806 06:31:29.598914 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 06:31:29.599057 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":79,"threads_started":1}
I20260806 06:31:29.599660 16840 heartbeater.cc:347] Connected to a master server at 127.7.43.126:36787
I20260806 06:31:29.599740 16840 heartbeater.cc:464] Registering TS with master...
I20260806 06:31:29.599889 16840 heartbeater.cc:511] Master 127.7.43.126:36787 requested a full tablet report, sending...
I20260806 06:31:29.600245 16142 ts_manager.cc:194] Registered new tserver with Master: 930504b65be34a37ac7455cda9c9c7b6 (127.7.43.67:41259)
I20260806 06:31:29.600582 16142 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.43.67@KRBTEST.COM'} at 127.7.43.67:39579
I20260806 06:31:29.604341 7341 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.7.43.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.43.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.43.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/oryx_127.7.43.125.keytab.
Entry for principal oryx/127.7.43.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/oryx_127.7.43.125.keytab.
I20260806 06:31:29.626606 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.125:34313
--webserver_interface=127.7.43.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/oryx_127.7.43.125.keytab
--principal=oryx/127.7.43.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:37673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.43.126:36787,127.7.43.125:34313
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:31:29.710109 16848 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:29.710296 16848 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:29.710326 16848 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:29.711484 16848 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:31:29.711525 16848 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:29.711544 16848 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:29.711560 16848 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:31:29.711577 16848 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:31:29.713055 16848 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:37673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.43.126:36787,127.7.43.125:34313
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/oryx_127.7.43.125.keytab
--principal=oryx/127.7.43.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.125:34313
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.43.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16848
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
W20260806 06:31:29.713166 16848 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.7.43.126:36787,127.7.43.125:34313'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260806 06:31:29.713304 16848 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:29.713508 16848 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:29.717928 16854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:29.717913 16857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:29.717904 16855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:29.718237 16848 server_base.cc:1034] running on GCE node
I20260806 06:31:29.718495 16848 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:29.718771 16848 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:29.719900 16848 hybrid_clock.cc:648] HybridClock initialized: now 1785997889719883 us; error 34 us; skew 500 ppm
Aug 06 06:31:29 dist-test-slave-6zbq krb5kdc[16073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997889, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.43.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:29.722152 16848 init.cc:377] Logged in from keytab as oryx/127.7.43.125@KRBTEST.COM (short username oryx)
I20260806 06:31:29.722595 16848 webserver.cc:460] Webserver started at http://127.7.43.125:42901/ using document root <none> and password file <none>
I20260806 06:31:29.722783 16848 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:29.722827 16848 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:29.722966 16848 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:29.723948 16848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-1/data/instance:
uuid: "bb5fccea882c4272838ffd2bb3d4a023"
format_stamp: "Formatted at 2026-08-06 06:31:29 on dist-test-slave-6zbq"
server_key: "71db13d0bad52a2c2c05cdaa9490020d"
server_key_iv: "5dee161ddbfc3cf5e07e74fef73248dc"
server_key_version: "encryptionkey@0"
I20260806 06:31:29.724282 16848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-1/wal/instance:
uuid: "bb5fccea882c4272838ffd2bb3d4a023"
format_stamp: "Formatted at 2026-08-06 06:31:29 on dist-test-slave-6zbq"
server_key: "71db13d0bad52a2c2c05cdaa9490020d"
server_key_iv: "5dee161ddbfc3cf5e07e74fef73248dc"
server_key_version: "encryptionkey@0"
I20260806 06:31:29.725481 16848 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 06:31:29.726101 16864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:29.726245 16848 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:29.726366 16848 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-1/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-1/wal
uuid: "bb5fccea882c4272838ffd2bb3d4a023"
format_stamp: "Formatted at 2026-08-06 06:31:29 on dist-test-slave-6zbq"
server_key: "71db13d0bad52a2c2c05cdaa9490020d"
server_key_iv: "5dee161ddbfc3cf5e07e74fef73248dc"
server_key_version: "encryptionkey@0"
I20260806 06:31:29.726437 16848 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:29.741640 16848 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:29.743386 16848 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:29.743506 16848 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:29.747179 16848 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.125:34313
I20260806 06:31:29.747210 16947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.125:34313 every 8 connection(s)
I20260806 06:31:29.747494 16848 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-1/data/info.pb
I20260806 06:31:29.748344 16951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:29.749675 16951 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:31:29.750629 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 16848
Aug 06 06:31:29 dist-test-slave-6zbq krb5kdc[16073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997889, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.43.125@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM
Aug 06 06:31:29 dist-test-slave-6zbq krb5kdc[16073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997888, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.43.125@KRBTEST.COM
I20260806 06:31:29.758780 16234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:29.750273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45614 (local address 127.7.43.126:36787)
0806 06:31:29.750333 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:29.750335 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:29.750530 (+ 195us) server_negotiation.cc:407] Connection header received
0806 06:31:29.751096 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:29.751097 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:29.751110 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:29.751165 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:29.751868 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:29.752612 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:29.753739 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:29.753939 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:29.753999 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:29.756136 (+ 2137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:29.756138 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:29.756138 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:29.756145 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:29.757424 (+ 1279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:29.757821 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:29.757822 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:29.757822 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:29.757865 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:29.758115 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:29.758115 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:29.758116 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:29.758306 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:29.758392 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:29.758657 (+ 265us) server_negotiation.cc:299] Negotiation successful
0806 06:31:29.758718 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 06:31:29.758976 16957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:29.750230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:36787 (local address 127.0.0.1:45614)
0806 06:31:29.750411 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:29.750453 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:29.750957 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:29.751260 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:29.751263 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:29.751558 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:29.751765 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:29.751771 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:29.752717 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:29.752718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:29.753661 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:29.753663 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:29.754019 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:29.754020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:29.754115 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:29.754577 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:29.754585 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:29.756045 (+ 1460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:29.757496 (+ 1451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:29.757501 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:29.757503 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:29.757752 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:29.757935 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:29.757935 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:29.757936 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:29.758055 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:29.758385 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:29.758390 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:29.758530 (+ 140us) client_negotiation.cc:769] Sending connection context
0806 06:31:29.758620 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 06:31:29.758761 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":88,"threads_started":1}
I20260806 06:31:29.759326 16959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:29.751034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40274 (local address 127.7.43.125:34313)
0806 06:31:29.751181 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:29.751185 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:29.751200 (+ 15us) server_negotiation.cc:407] Connection header received
0806 06:31:29.751252 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:29.751254 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:29.751268 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:29.751343 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:29.752022 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:29.753038 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:29.753931 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:29.754277 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:29.754325 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:29.756493 (+ 2168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:29.756500 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:29.756501 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:29.756507 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:29.758078 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:29.758428 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:29.758431 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:29.758431 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:29.758488 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:29.758653 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:29.758653 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:29.758654 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:29.758797 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:29.758925 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:29.759144 (+ 219us) server_negotiation.cc:299] Negotiation successful
0806 06:31:29.759220 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":76,"threads_started":1}
I20260806 06:31:29.759610 16951 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.43.125" port: 34313 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:31:29 dist-test-slave-6zbq krb5kdc[16073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997889, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.43.125@KRBTEST.COM for oryx/127.7.43.125@KRBTEST.COM
I20260806 06:31:29.765326 16957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:29.759816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.125:34313 (local address 127.0.0.1:40288)
0806 06:31:29.759858 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:29.759868 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:29.759901 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:29.760012 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:29.760013 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:29.760109 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:29.760151 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:29.760152 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:29.760735 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:29.760736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:29.761205 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:29.761207 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:29.761536 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:29.761537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:29.761584 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:29.762143 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:29.762147 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:29.763159 (+ 1012us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:29.764500 (+ 1341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:29.764501 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:29.764501 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:29.764642 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:29.764791 (+ 149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:29.764792 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:29.764792 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:29.764850 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:29.765038 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:29.765039 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:29.765085 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 06:31:29.765142 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 06:31:29.765229 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 06:31:29.765377 16959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:29.759860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40288 (local address 127.7.43.125:34313)
0806 06:31:29.759891 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:29.759893 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:29.759900 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:31:29.759959 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:29.759959 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:29.759968 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:29.760009 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:29.760216 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:29.760660 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:29.761274 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:29.761482 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:29.761532 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:29.763230 (+ 1698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:29.763231 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:29.763231 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:29.763237 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:29.764422 (+ 1185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:29.764696 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:29.764697 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:29.764697 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:29.764742 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:29.764898 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:29.764899 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:29.764899 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:29.764987 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:29.765037 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:29.765168 (+ 131us) server_negotiation.cc:299] Negotiation successful
0806 06:31:29.765274 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":18}
I20260806 06:31:29.766759 16951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023: Bootstrap starting.
I20260806 06:31:29.767506 16951 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:29.767750 16951 log.cc:824] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023: Log is configured to *not* fsync() on all Append() calls
I20260806 06:31:29.768381 16951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023: No bootstrap required, opened a new log
I20260806 06:31:29.769651 16951 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } } peers { permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 34313 } }
I20260806 06:31:29.769759 16951 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:29.769793 16951 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb5fccea882c4272838ffd2bb3d4a023, State: Initialized, Role: FOLLOWER
I20260806 06:31:29.769906 16951 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } } peers { permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 34313 } }
I20260806 06:31:29.770116 16961 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } } peers { permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 34313 } } }
I20260806 06:31:29.770161 16951 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:31:29.770200 16961 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 06:31:29.773198 16978 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:31:29.773254 16978 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:31:29.774837 16848 master_runner.cc:428] Detected that this master bb5fccea882c4272838ffd2bb3d4a023 is joining an existing cluster
I20260806 06:31:29.774910 16848 master_runner.cc:432] Initiating AddMaster RPC to add 127.7.43.125:34313
I20260806 06:31:29.781850 16234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:29.775888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45630 (local address 127.7.43.126:36787)
0806 06:31:29.775928 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:29.775929 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:29.776029 (+ 100us) server_negotiation.cc:407] Connection header received
0806 06:31:29.776129 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:29.776130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:29.776140 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:29.776169 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:29.776669 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:29.777377 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:29.778069 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:29.778268 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:29.778320 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:29.779366 (+ 1046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:29.779367 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:29.779368 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:29.779374 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:29.780684 (+ 1310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:29.781116 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:29.781117 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:29.781117 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:29.781173 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:29.781365 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:29.781366 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:29.781367 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:29.781485 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:29.781519 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:29.781728 (+ 209us) server_negotiation.cc:299] Negotiation successful
0806 06:31:29.781791 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 06:31:29.782124 16989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:29.775808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:36787 (local address 127.0.0.1:45630)
0806 06:31:29.775950 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:29.775960 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:29.776059 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:29.776226 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:29.776226 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:29.776482 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:29.776570 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:29.776572 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:29.777431 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:29.777431 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:29.777995 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:29.777996 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:29.778400 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:29.778401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:29.778458 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:29.778858 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:29.778861 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:29.779267 (+ 406us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:29.780758 (+ 1491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:29.780759 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:29.780759 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:29.781040 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:29.781242 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:29.781242 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:29.781243 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:29.781294 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:29.781561 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:29.781562 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:29.781610 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 06:31:29.781925 (+ 315us) client_negotiation.cc:240] Negotiation successful
0806 06:31:29.782013 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":65,"threads_started":1}
I20260806 06:31:29.782202 16959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:29.776020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40296 (local address 127.7.43.125:34313)
0806 06:31:29.776052 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:29.776053 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:29.776173 (+ 120us) server_negotiation.cc:407] Connection header received
0806 06:31:29.776221 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:29.776222 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:29.776230 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:29.776275 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:29.776615 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:29.777305 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:29.778299 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:29.778554 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:29.778621 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:29.779443 (+ 822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:29.779444 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:29.779444 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:29.779451 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:29.781116 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:29.781435 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:29.781437 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:29.781437 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:29.781496 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:29.781694 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:29.781695 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:29.781695 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:29.781811 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:29.781876 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:29.782076 (+ 200us) server_negotiation.cc:299] Negotiation successful
0806 06:31:29.782135 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 06:31:29.782472 16991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:29.775998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.125:34313 (local address 127.0.0.1:40296)
0806 06:31:29.776115 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:29.776123 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:29.776171 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:29.776283 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:29.776284 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:29.776482 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:29.776548 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:29.776550 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:29.777385 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:29.777386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:29.778212 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:29.778215 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:29.778655 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:29.778656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:29.778696 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:29.778996 (+ 300us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:29.779000 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:29.779362 (+ 362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:29.781189 (+ 1827us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:29.781190 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:29.781190 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:29.781372 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:29.781561 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:29.781561 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:29.781562 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:29.781637 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:29.781878 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:29.781879 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:29.781966 (+ 87us) client_negotiation.cc:769] Sending connection context
0806 06:31:29.782276 (+ 310us) client_negotiation.cc:240] Negotiation successful
0806 06:31:29.782374 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":59,"threads_started":1}
Aug 06 06:31:29 dist-test-slave-6zbq krb5kdc[16073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997889, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.43.126@KRBTEST.COM for oryx/127.7.43.125@KRBTEST.COM
I20260806 06:31:29.790709 16959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:29.784171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40306 (local address 127.7.43.125:34313)
0806 06:31:29.784205 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:29.784207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:29.784340 (+ 133us) server_negotiation.cc:407] Connection header received
0806 06:31:29.784374 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:29.784375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:29.784384 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:29.784444 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:29.784771 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:29.785454 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:29.786134 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:29.786456 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:29.786524 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:29.788479 (+ 1955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:29.788480 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:29.788481 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:29.788486 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:29.789634 (+ 1148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:29.790043 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:29.790043 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:29.790043 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:29.790082 (+ 39us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:29.790269 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:29.790270 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:29.790270 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:29.790390 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:29.790439 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:29.790610 (+ 171us) server_negotiation.cc:299] Negotiation successful
0806 06:31:29.790659 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 06:31:29.790784 16992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:29.784091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.125:34313 (local address 127.0.0.1:40306)
0806 06:31:29.784240 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:29.784269 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:29.784330 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:29.784448 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:29.784450 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:29.784671 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:29.784711 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:29.784712 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:29.785524 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:29.785525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:29.786063 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:29.786065 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:29.786521 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:29.786521 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:29.786570 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:29.786955 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:29.786959 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:29.788403 (+ 1444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:29.789711 (+ 1308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:29.789720 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:29.789722 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:29.789976 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:29.790149 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:29.790150 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:29.790150 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:29.790216 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:29.790435 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:29.790442 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:29.790492 (+ 50us) client_negotiation.cc:769] Sending connection context
0806 06:31:29.790576 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 06:31:29.790685 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":64,"threads_started":1}
I20260806 06:31:29.791280 16142 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.7.43.125:34313: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023"
member_type: NON_VOTER
last_known_addr {
host: "127.7.43.125"
port: 34313
}
attrs {
promote: true
}
}
dest_uuid: "ee7ac2b4afd849459e289fe5dba19836"
cas_config_opid_index: -1
I20260806 06:31:29.791554 16142 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } } peers { permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023" member_type: NON_VOTER last_known_addr { host: "127.7.43.125" port: 34313 } attrs { promote: true } }
I20260806 06:31:29.791960 16993 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER bb5fccea882c4272838ffd2bb3d4a023 (127.7.43.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } } peers { permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023" member_type: NON_VOTER last_known_addr { host: "127.7.43.125" port: 34313 } attrs { promote: true } } }
I20260806 06:31:29.792235 16909 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:29.862864 16909 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023 [term 1 FOLLOWER]: Refusing update from remote peer ee7ac2b4afd849459e289fe5dba19836: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 06:31:29.862910 16993 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.7.43.125:34313
I20260806 06:31:29.863021 16993 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ee7ac2b4afd849459e289fe5dba19836" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } } peers { permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023" member_type: NON_VOTER last_known_addr { host: "127.7.43.125" port: 34313 } attrs { promote: true } } }
I20260806 06:31:29.863092 16993 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836 [sys.catalog]: This master's current role is: LEADER
I20260806 06:31:29.865479 16993 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023" member_type: NON_VOTER last_known_addr { host: "127.7.43.125" port: 34313 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:31:29.867525 16909 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023 [term 1 LEARNER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER bb5fccea882c4272838ffd2bb3d4a023 (127.7.43.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } } peers { permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023" member_type: NON_VOTER last_known_addr { host: "127.7.43.125" port: 34313 } attrs { promote: true } } }
I20260806 06:31:29.868577 16848 master.cc:537] Master@127.7.43.125:34313 shutting down...
I20260806 06:31:29.868995 16999 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ee7ac2b4afd849459e289fe5dba19836" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } } peers { permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023" member_type: NON_VOTER last_known_addr { host: "127.7.43.125" port: 34313 } attrs { promote: true } } }
I20260806 06:31:29.869084 16999 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023 [sys.catalog]: This master's current role is: LEARNER
I20260806 06:31:29.869130 16999 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ee7ac2b4afd849459e289fe5dba19836" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } } peers { permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023" member_type: NON_VOTER last_known_addr { host: "127.7.43.125" port: 34313 } attrs { promote: true } } }
I20260806 06:31:29.869171 16999 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023 [sys.catalog]: This master's current role is: LEARNER
W20260806 06:31:29.869421 16961 raft_consensus.cc:659] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } } peers { permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023" member_type: NON_VOTER last_known_addr { host: "127.7.43.125" port: 34313 } attrs { promote: true } }
I20260806 06:31:29.869922 17004 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee7ac2b4afd849459e289fe5dba19836. Latest consensus state: current_term: 1 leader_uuid: "ee7ac2b4afd849459e289fe5dba19836" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } } peers { permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023" member_type: NON_VOTER last_known_addr { host: "127.7.43.125" port: 34313 } attrs { promote: true } } }
I20260806 06:31:29.870013 17004 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023 [sys.catalog]: This master's current role is: LEARNER
I20260806 06:31:29.870319 17000 mvcc.cc:205] Tried to move back new op lower bound from 7315447353969491968 to 7315447353878417408. Current Snapshot: MvccSnapshot[applied={T|T < 7315447353885872128}]
I20260806 06:31:29.872314 16848 raft_consensus.cc:2229] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023 [term 1 LEARNER]: Raft consensus shutting down.
I20260806 06:31:29.872427 16848 raft_consensus.cc:2258] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023 [term 1 LEARNER]: Raft consensus is shut down!
I20260806 06:31:29.872468 16848 tablet_replica.cc:330] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023: stopping tablet replica
WARNING: no policy specified for oryx/127.7.43.124@KRBTEST.COM; defaulting to no policy
I20260806 06:31:29.885586 16848 master.cc:552] Master@127.7.43.125:34313 shutdown complete.
I20260806 06:31:29.885625 16848 master_runner.cc:305] Clearing existing system tablet
I20260806 06:31:29.885736 16848 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:29.885776 16848 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:29.886298 16848 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.002s
I20260806 06:31:29.886739 17016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:29.886842 16848 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:29.886904 16848 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:29.886924 16848 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-1/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-1/wal
uuid: "bb5fccea882c4272838ffd2bb3d4a023"
format_stamp: "Formatted at 2026-08-06 06:31:29 on dist-test-slave-6zbq"
server_key: "71db13d0bad52a2c2c05cdaa9490020d"
server_key_iv: "5dee161ddbfc3cf5e07e74fef73248dc"
server_key_version: "encryptionkey@0"
I20260806 06:31:29.887539 16848 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:31:30.005223 16848 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 06:31:30.005337 16848 log.cc:1196] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260806 06:31:30.005574 16848 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023: Deleting consensus metadata
I20260806 06:31:30.005687 16848 master_runner.cc:315] Copying system tablet from 127.7.43.126:36787
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.43.124@KRBTEST.COM" created.
I20260806 06:31:30.006318 16848 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023: tablet copy: Beginning tablet copy session from remote peer at address 127.7.43.126:36787
I20260806 06:31:30.012626 17025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:30.006711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:36787 (local address 127.0.0.1:45640)
0806 06:31:30.006852 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:30.006865 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:30.006938 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:30.007246 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:30.007247 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:30.007441 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:30.007491 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:30.007493 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:30.008228 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:30.008229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:30.009047 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:30.009049 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:30.009410 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:30.009411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:30.009477 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:30.009910 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:30.009916 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:30.010209 (+ 293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:30.011693 (+ 1484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:30.011694 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:30.011695 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:30.011869 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:30.012038 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:30.012039 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:30.012039 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:30.012097 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:30.012302 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:30.012303 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:30.012358 (+ 55us) client_negotiation.cc:769] Sending connection context
0806 06:31:30.012416 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 06:31:30.012523 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":76,"threads_started":1}
I20260806 06:31:30.012619 16234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:30.007039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45640 (local address 127.7.43.126:36787)
0806 06:31:30.007107 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:30.007110 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:30.007120 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:31:30.007152 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:30.007153 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:30.007164 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:30.007225 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:30.007570 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:30.008154 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:30.009123 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:30.009326 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:30.009380 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:30.010299 (+ 919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:30.010301 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:30.010301 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:30.010309 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:30.011620 (+ 1311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:30.011933 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:30.011934 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:30.011934 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:30.011986 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:30.012147 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:30.012148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:30.012148 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:30.012248 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:30.012303 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:30.012445 (+ 142us) server_negotiation.cc:299] Negotiation successful
0806 06:31:30.012523 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 06:31:30.013082 16172 tablet_copy_service.cc:140] P ee7ac2b4afd849459e289fe5dba19836: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer bb5fccea882c4272838ffd2bb3d4a023 ({username='oryx', principal='oryx/127.7.43.125@KRBTEST.COM'} at 127.0.0.1:45640)
I20260806 06:31:30.013197 16172 tablet_copy_service.cc:161] P ee7ac2b4afd849459e289fe5dba19836: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer bb5fccea882c4272838ffd2bb3d4a023 at {username='oryx', principal='oryx/127.7.43.125@KRBTEST.COM'} at 127.0.0.1:45640: session id = bb5fccea882c4272838ffd2bb3d4a023-00000000000000000000000000000000
I20260806 06:31:30.013999 16172 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 06:31:30.014678 16848 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 06:31:30.015861 16848 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023: tablet copy: Starting download of 0 data blocks...
I20260806 06:31:30.015981 16848 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023: tablet copy: Starting download of 1 WAL segments...
I20260806 06:31:30.016816 16848 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 06:31:30.105861 16172 tablet_copy_service.cc:342] P ee7ac2b4afd849459e289fe5dba19836: Request end of tablet copy session bb5fccea882c4272838ffd2bb3d4a023-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.7.43.125@KRBTEST.COM'} at 127.0.0.1:45640
I20260806 06:31:30.105929 16172 tablet_copy_service.cc:433] P ee7ac2b4afd849459e289fe5dba19836: ending tablet copy session bb5fccea882c4272838ffd2bb3d4a023-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer bb5fccea882c4272838ffd2bb3d4a023
I20260806 06:31:30.108654 16848 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:30.109615 17029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:30.109710 17035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:30.109759 17030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:30.109977 16848 server_base.cc:1034] running on GCE node
I20260806 06:31:30.110039 16848 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:30.110148 16848 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:30.111258 16848 hybrid_clock.cc:648] HybridClock initialized: now 1785997890111247 us; error 24 us; skew 500 ppm
Aug 06 06:31:30 dist-test-slave-6zbq krb5kdc[16073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997890, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.43.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:30.173483 16848 init.cc:377] Logged in from keytab as oryx/127.7.43.125@KRBTEST.COM (short username oryx)
I20260806 06:31:30.173745 16848 webserver.cc:460] Webserver started at http://127.7.43.125:39301/ using document root <none> and password file <none>
I20260806 06:31:30.173864 16848 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:30.173898 16848 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:30.174544 16848 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:30.175038 17045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:30.175169 16848 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:31:30.175230 16848 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-1/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-1/wal
uuid: "bb5fccea882c4272838ffd2bb3d4a023"
format_stamp: "Formatted at 2026-08-06 06:31:29 on dist-test-slave-6zbq"
server_key: "71db13d0bad52a2c2c05cdaa9490020d"
server_key_iv: "5dee161ddbfc3cf5e07e74fef73248dc"
server_key_version: "encryptionkey@0"
I20260806 06:31:30.175292 16848 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:30.191917 16848 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:30.193360 16848 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:30.195779 16848 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.125:34313
I20260806 06:31:30.196488 17113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.125:34313 every 8 connection(s)
I20260806 06:31:30.273038 16848 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-1/data/info.pb
I20260806 06:31:30.274775 17114 sys_catalog.cc:263] Verifying existing consensus state
I20260806 06:31:30.275331 17114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023: Bootstrap starting.
I20260806 06:31:30.277521 17114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:31:30.277710 17114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023: Bootstrap complete.
I20260806 06:31:30.277891 17114 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } } peers { permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023" member_type: NON_VOTER last_known_addr { host: "127.7.43.125" port: 34313 } attrs { promote: true } }
I20260806 06:31:30.277963 17114 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: bb5fccea882c4272838ffd2bb3d4a023, State: Initialized, Role: LEARNER
I20260806 06:31:30.277993 17114 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } } peers { permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023" member_type: NON_VOTER last_known_addr { host: "127.7.43.125" port: 34313 } attrs { promote: true } }
I20260806 06:31:30.278162 17114 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:31:30.278153 17120 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } } peers { permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023" member_type: NON_VOTER last_known_addr { host: "127.7.43.125" port: 34313 } attrs { promote: true } } }
I20260806 06:31:30.278251 17120 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023 [sys.catalog]: This master's current role is: LEARNER
I20260806 06:31:30.279372 17137 catalog_manager.cc:1259] Loaded cluster ID: f88ac3451d944ee6994511cbff6c6a75
I20260806 06:31:30.279419 17137 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023: loading cluster ID for follower catalog manager: success
I20260806 06:31:30.280381 17137 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023: acquiring CA information for follower catalog manager: success
I20260806 06:31:30.280617 17137 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 06:31:30.317962 16427 heartbeater.cc:503] Master 127.7.43.126:36787 was elected leader, sending a full tablet report...
I20260806 06:31:30.389927 17141 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ee7ac2b4afd849459e289fe5dba19836" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } } peers { permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023" member_type: NON_VOTER last_known_addr { host: "127.7.43.125" port: 34313 } attrs { promote: true } } }
I20260806 06:31:30.390086 17141 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836 [sys.catalog]: This master's current role is: LEADER
I20260806 06:31:30.392988 17149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:30.390132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40310 (local address 127.7.43.125:34313)
0806 06:31:30.390251 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:30.390253 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:30.390264 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:31:30.390319 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:30.390320 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:30.390369 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:30.390435 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:31:30.390589 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:30.391242 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:30.392197 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:30.392623 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:30.392688 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:30.392771 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:30.392872 (+ 101us) server_negotiation.cc:299] Negotiation successful
0806 06:31:30.392916 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":54,"threads_started":1}
I20260806 06:31:30.392988 17142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:30.390067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.125:34313 (local address 127.0.0.1:40310)
0806 06:31:30.390183 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:30.390193 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:30.390265 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:30.390468 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:30.390469 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:30.390472 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:31:30.390525 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:30.390527 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:30.391347 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:30.391348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:30.392104 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:30.392106 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:30.392707 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:30.392708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:30.392776 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:30.392805 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 06:31:30.392858 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 06:31:30.392909 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":59,"threads_started":1}
I20260806 06:31:30.393330 17120 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee7ac2b4afd849459e289fe5dba19836. Latest consensus state: current_term: 1 leader_uuid: "ee7ac2b4afd849459e289fe5dba19836" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } } peers { permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023" member_type: NON_VOTER last_known_addr { host: "127.7.43.125" port: 34313 } attrs { promote: true } } }
I20260806 06:31:30.393427 17120 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023 [sys.catalog]: This master's current role is: LEARNER
I20260806 06:31:30.393582 17141 raft_consensus.cc:1050] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836: attempting to promote NON_VOTER bb5fccea882c4272838ffd2bb3d4a023 to VOTER
I20260806 06:31:30.393787 17141 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } } peers { permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 34313 } attrs { promote: false } }
I20260806 06:31:30.394156 17084 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023 [term 1 LEARNER]: Refusing update from remote peer ee7ac2b4afd849459e289fe5dba19836: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 06:31:30.394358 17141 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 34313 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 06:31:30.412046 17140 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bb5fccea882c4272838ffd2bb3d4a023 (127.7.43.125) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } } peers { permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 34313 } attrs { promote: false } } }
I20260806 06:31:30.412266 17084 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bb5fccea882c4272838ffd2bb3d4a023 (127.7.43.125) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } } peers { permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 34313 } attrs { promote: false } } }
I20260806 06:31:30.412943 17140 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ee7ac2b4afd849459e289fe5dba19836" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } } peers { permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 34313 } attrs { promote: false } } }
I20260806 06:31:30.413019 17140 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836 [sys.catalog]: This master's current role is: LEADER
I20260806 06:31:30.413061 17140 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ee7ac2b4afd849459e289fe5dba19836" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } } peers { permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 34313 } attrs { promote: false } } }
I20260806 06:31:30.413100 17140 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836 [sys.catalog]: This master's current role is: LEADER
I20260806 06:31:30.413357 17120 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ee7ac2b4afd849459e289fe5dba19836" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } } peers { permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 34313 } attrs { promote: false } } }
I20260806 06:31:30.413439 17120 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:31:30.462677 16628 heartbeater.cc:503] Master 127.7.43.126:36787 was elected leader, sending a full tablet report...
I20260806 06:31:30.601290 16840 heartbeater.cc:503] Master 127.7.43.126:36787 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.43.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/oryx_127.7.43.124.keytab.
Entry for principal oryx/127.7.43.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/oryx_127.7.43.124.keytab.
W20260806 06:31:30.716094 7341 mini_kdc.cc:262] Time spent creating service keytab for oryx/127.7.43.124: real 0.852s user 0.000s sys 0.004s
I20260806 06:31:30.716445 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.124:45905
--webserver_interface=127.7.43.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/oryx_127.7.43.124.keytab
--principal=oryx/127.7.43.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:37673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.43.126:36787,127.7.43.125:34313,127.7.43.124:45905
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:31:30.801932 17152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:30.802106 17152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:30.802126 17152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:30.803259 17152 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:31:30.803298 17152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:30.803310 17152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:30.803324 17152 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:31:30.803349 17152 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:31:30.804785 17152 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:37673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.43.126:36787,127.7.43.125:34313,127.7.43.124:45905
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/oryx_127.7.43.124.keytab
--principal=oryx/127.7.43.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.124:45905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.43.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17152
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:30.804999 17152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:30.805186 17152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:30.809526 17162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:30.809614 17152 server_base.cc:1034] running on GCE node
W20260806 06:31:30.809530 17160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:30.809541 17158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:30.809962 17152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:30.810205 17152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:30.811340 17152 hybrid_clock.cc:648] HybridClock initialized: now 1785997890811331 us; error 27 us; skew 500 ppm
Aug 06 06:31:30 dist-test-slave-6zbq krb5kdc[16073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997890, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.43.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:30.816679 17152 init.cc:377] Logged in from keytab as oryx/127.7.43.124@KRBTEST.COM (short username oryx)
I20260806 06:31:30.817066 17152 webserver.cc:460] Webserver started at http://127.7.43.124:40685/ using document root <none> and password file <none>
I20260806 06:31:30.817274 17152 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:30.817322 17152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:30.817451 17152 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:30.818480 17152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-2/data/instance:
uuid: "cbb47ccf5ac7455dbb8898888f83dbf3"
format_stamp: "Formatted at 2026-08-06 06:31:30 on dist-test-slave-6zbq"
server_key: "7a3b66a886cf07f2f4004d5c0bcd0c21"
server_key_iv: "c121b16cc1e7a800325d4a98c4397d4f"
server_key_version: "encryptionkey@0"
I20260806 06:31:30.818794 17152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-2/wal/instance:
uuid: "cbb47ccf5ac7455dbb8898888f83dbf3"
format_stamp: "Formatted at 2026-08-06 06:31:30 on dist-test-slave-6zbq"
server_key: "7a3b66a886cf07f2f4004d5c0bcd0c21"
server_key_iv: "c121b16cc1e7a800325d4a98c4397d4f"
server_key_version: "encryptionkey@0"
I20260806 06:31:30.820005 17152 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:31:30.820640 17173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:30.820783 17152 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:31:30.820878 17152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-2/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-2/wal
uuid: "cbb47ccf5ac7455dbb8898888f83dbf3"
format_stamp: "Formatted at 2026-08-06 06:31:30 on dist-test-slave-6zbq"
server_key: "7a3b66a886cf07f2f4004d5c0bcd0c21"
server_key_iv: "c121b16cc1e7a800325d4a98c4397d4f"
server_key_version: "encryptionkey@0"
I20260806 06:31:30.820945 17152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:30.830592 17152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:30.832409 17152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:30.832535 17152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:30.837944 17152 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.124:45905
I20260806 06:31:30.838063 17249 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.124:45905 every 8 connection(s)
I20260806 06:31:30.838299 17152 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-2/data/info.pb
I20260806 06:31:30.839288 17250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:30.840814 17250 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:31:30.841905 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 17152
Aug 06 06:31:30 dist-test-slave-6zbq krb5kdc[16073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997890, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.43.124@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM
Aug 06 06:31:30 dist-test-slave-6zbq krb5kdc[16073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997888, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.43.124@KRBTEST.COM
I20260806 06:31:30.850494 17253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:30.841511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45644 (local address 127.7.43.126:36787)
0806 06:31:30.841630 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:30.841632 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:30.841844 (+ 212us) server_negotiation.cc:407] Connection header received
0806 06:31:30.842539 (+ 695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:30.842541 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:30.842558 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:30.842622 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:30.843166 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:30.843799 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:30.845013 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:30.845328 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:30.845391 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:30.847420 (+ 2029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:30.847421 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:30.847421 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:30.847428 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:30.849130 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:30.849586 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:30.849587 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:30.849587 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:30.849644 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:30.849862 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:30.849863 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:30.849863 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:30.849975 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:30.850031 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:30.850289 (+ 258us) server_negotiation.cc:299] Negotiation successful
0806 06:31:30.850400 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":57,"threads_started":1}
I20260806 06:31:30.850607 17254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:30.841390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:36787 (local address 127.0.0.1:45644)
0806 06:31:30.841691 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:30.841752 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:30.842423 (+ 671us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:30.842700 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:30.842703 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:30.842925 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:30.843069 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:30.843073 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:30.843903 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:30.843904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:30.844944 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:30.844946 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:30.845401 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:30.845402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:30.845499 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:30.845927 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:30.845935 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:30.847333 (+ 1398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:30.849207 (+ 1874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:30.849214 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:30.849216 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:30.849461 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:30.849700 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:30.849701 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:30.849701 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:30.849802 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:30.850031 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:30.850038 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:30.850162 (+ 124us) client_negotiation.cc:769] Sending connection context
0806 06:31:30.850247 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 06:31:30.850409 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":175,"threads_started":1}
I20260806 06:31:30.850620 17259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:30.842433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58696 (local address 127.7.43.124:45905)
0806 06:31:30.842715 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:30.842718 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:30.842734 (+ 16us) server_negotiation.cc:407] Connection header received
0806 06:31:30.842751 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:30.842752 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:30.842769 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:30.842850 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:30.843511 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:30.844828 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:30.845681 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:30.845973 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:30.846029 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:30.848172 (+ 2143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:30.848179 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:30.848180 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:30.848186 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:30.849593 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:30.849892 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:30.849895 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:30.849895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:30.849945 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:30.850105 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:30.850106 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:30.850106 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:30.850229 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:30.850380 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:30.850456 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 06:31:30.850526 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":170,"threads_started":1}
I20260806 06:31:30.851238 17250 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.43.125" port: 34313 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:31:30 dist-test-slave-6zbq krb5kdc[16073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997890, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.43.124@KRBTEST.COM for oryx/127.7.43.125@KRBTEST.COM
I20260806 06:31:30.856375 17254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:30.851438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.125:34313 (local address 127.0.0.1:40322)
0806 06:31:30.851477 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:30.851486 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:30.851519 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:30.851627 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:30.851627 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:30.851726 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:30.851767 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:30.851768 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:30.852302 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:30.852303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:30.852761 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:30.852763 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:30.853028 (+ 265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:30.853029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:30.853074 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:30.853357 (+ 283us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:30.853360 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:30.854315 (+ 955us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:30.855711 (+ 1396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:30.855711 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:30.855712 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:30.855849 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:30.855983 (+ 134us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:30.855984 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:30.855984 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:30.856025 (+ 41us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:30.856177 (+ 152us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:30.856177 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:30.856207 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 06:31:30.856249 (+ 42us) client_negotiation.cc:240] Negotiation successful
0806 06:31:30.856309 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260806 06:31:30.856386 17149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:30.851501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40322 (local address 127.7.43.125:34313)
0806 06:31:30.851540 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:30.851542 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:30.851549 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:31:30.851572 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:30.851573 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:30.851582 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:30.851624 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:30.851806 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:30.852254 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:30.852815 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:30.852985 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:30.853024 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:30.854411 (+ 1387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:30.854413 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:30.854413 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:30.854421 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:30.855649 (+ 1228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:30.855899 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:30.855899 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:30.855900 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:30.855944 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:30.856058 (+ 114us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:30.856058 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:30.856059 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:30.856142 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:30.856179 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:30.856267 (+ 88us) server_negotiation.cc:299] Negotiation successful
0806 06:31:30.856327 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:31:30.856717 17250 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.43.124" port: 45905 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:31:30 dist-test-slave-6zbq krb5kdc[16073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997890, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.43.124@KRBTEST.COM for oryx/127.7.43.124@KRBTEST.COM
I20260806 06:31:30.861810 17254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:30.856929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.124:45905 (local address 127.0.0.1:58712)
0806 06:31:30.856966 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:30.856974 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:30.857004 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:30.857092 (+ 88us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:30.857093 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:30.857189 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:30.857213 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:30.857214 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:30.857801 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:30.857801 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:30.858226 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:30.858227 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:30.858613 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:30.858613 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:30.858647 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:30.858954 (+ 307us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:30.858959 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:30.859903 (+ 944us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:30.861127 (+ 1224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:30.861128 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:30.861128 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:30.861275 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:30.861425 (+ 150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:30.861425 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:30.861425 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:30.861466 (+ 41us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:30.861614 (+ 148us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:30.861615 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:30.861643 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 06:31:30.861686 (+ 43us) client_negotiation.cc:240] Negotiation successful
0806 06:31:30.861756 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260806 06:31:30.861881 17259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:30.856960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58712 (local address 127.7.43.124:45905)
0806 06:31:30.856988 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:30.856990 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:30.856998 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:31:30.857039 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:30.857039 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:30.857046 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:30.857085 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:30.857258 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:30.857744 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:30.858275 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:30.858553 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:30.858617 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:30.859976 (+ 1359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:30.859977 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:30.859977 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:30.859982 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:30.861060 (+ 1078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:30.861329 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:30.861330 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:30.861330 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:30.861381 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:30.861499 (+ 118us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:30.861500 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:30.861500 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:30.861577 (+ 77us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:30.861617 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:30.861701 (+ 84us) server_negotiation.cc:299] Negotiation successful
0806 06:31:30.861798 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":13}
I20260806 06:31:30.863040 17250 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3: Bootstrap starting.
I20260806 06:31:30.864066 17250 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:30.864432 17250 log.cc:824] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3: Log is configured to *not* fsync() on all Append() calls
I20260806 06:31:30.865056 17250 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3: No bootstrap required, opened a new log
I20260806 06:31:30.866890 17250 raft_consensus.cc:348] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } } peers { permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 34313 } } peers { permanent_uuid: "cbb47ccf5ac7455dbb8898888f83dbf3" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 45905 } }
I20260806 06:31:30.867030 17250 raft_consensus.cc:374] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:30.867063 17250 raft_consensus.cc:729] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbb47ccf5ac7455dbb8898888f83dbf3, State: Initialized, Role: FOLLOWER
I20260806 06:31:30.867177 17250 consensus_queue.cc:260] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } } peers { permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 34313 } } peers { permanent_uuid: "cbb47ccf5ac7455dbb8898888f83dbf3" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 45905 } }
I20260806 06:31:30.867506 17250 sys_catalog.cc:564] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:31:30.867539 17263 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } } peers { permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 34313 } } peers { permanent_uuid: "cbb47ccf5ac7455dbb8898888f83dbf3" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 45905 } } }
I20260806 06:31:30.867659 17263 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 06:31:30.869477 17278 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:31:30.869532 17278 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:31:30.871551 17152 master_runner.cc:428] Detected that this master cbb47ccf5ac7455dbb8898888f83dbf3 is joining an existing cluster
I20260806 06:31:30.871603 17152 master_runner.cc:432] Initiating AddMaster RPC to add 127.7.43.124:45905
I20260806 06:31:30.879377 17253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:30.872735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45656 (local address 127.7.43.126:36787)
0806 06:31:30.872782 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:30.872784 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:30.873083 (+ 299us) server_negotiation.cc:407] Connection header received
0806 06:31:30.873129 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:30.873130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:30.873142 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:30.873192 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:30.873586 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:30.874305 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:30.875522 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:30.875807 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:30.875852 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:30.876887 (+ 1035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:30.876888 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:30.876888 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:30.876893 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:30.878233 (+ 1340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:30.878611 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:30.878611 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:30.878612 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:30.878677 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:30.878855 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:30.878855 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:30.878855 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:30.878955 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:30.879016 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:30.879235 (+ 219us) server_negotiation.cc:299] Negotiation successful
0806 06:31:30.879304 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:31:30.879582 17286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:30.872719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:36787 (local address 127.0.0.1:45656)
0806 06:31:30.873011 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:30.873019 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:30.873074 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:30.873246 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:30.873247 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:30.873421 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:30.873494 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:30.873496 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:30.874428 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:30.874428 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:30.875426 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:30.875429 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:30.875928 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:30.875929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:30.875994 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:30.876409 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:30.876413 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:30.876803 (+ 390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:30.878303 (+ 1500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:30.878305 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:30.878305 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:30.878541 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:30.878734 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:30.878735 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:30.878735 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:30.878799 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:30.879043 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:30.879044 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:30.879100 (+ 56us) client_negotiation.cc:769] Sending connection context
0806 06:31:30.879198 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 06:31:30.879301 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":267,"threads_started":1}
Aug 06 06:31:30 dist-test-slave-6zbq krb5kdc[16073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997889, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.43.126@KRBTEST.COM for oryx/127.7.43.124@KRBTEST.COM
I20260806 06:31:30.897675 17142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:30.881487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.124:45905 (local address 127.0.0.1:58722)
0806 06:31:30.881534 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:30.881544 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:30.881587 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:30.881789 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:30.881789 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:30.881944 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:30.881974 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:30.881976 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:30.882646 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:30.882647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:30.883187 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:30.883189 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:30.883584 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:30.883586 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:30.883641 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:30.883974 (+ 333us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:30.883979 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:30.885261 (+ 1282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:30.896653 (+ 11392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:30.896654 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:30.896654 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:30.896854 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:30.897061 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:30.897062 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:30.897062 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:30.897120 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:30.897387 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:30.897388 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:30.897433 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 06:31:30.897479 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 06:31:30.897576 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 06:31:30.897998 17289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:30.881532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58722 (local address 127.7.43.124:45905)
0806 06:31:30.881684 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:30.881685 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:30.881691 (+ 6us) server_negotiation.cc:407] Connection header received
0806 06:31:30.881717 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:30.881717 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:30.881725 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:30.881777 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:30.882050 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:30.882579 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:30.883265 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:30.883518 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:30.883574 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:30.885342 (+ 1768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:30.885344 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:30.885344 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:30.885350 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:30.896557 (+ 11207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:30.896935 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:30.896935 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:30.896936 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:30.896990 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:30.897183 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:30.897184 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:30.897184 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:30.897309 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:30.897782 (+ 473us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:30.897822 (+ 40us) server_negotiation.cc:299] Negotiation successful
0806 06:31:30.897901 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":83,"threads_started":1}
I20260806 06:31:30.898520 16142 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.7.43.124:45905: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "cbb47ccf5ac7455dbb8898888f83dbf3"
member_type: NON_VOTER
last_known_addr {
host: "127.7.43.124"
port: 45905
}
attrs {
promote: true
}
}
dest_uuid: "ee7ac2b4afd849459e289fe5dba19836"
cas_config_opid_index: 6
I20260806 06:31:30.898624 17149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:30.873136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40334 (local address 127.7.43.125:34313)
0806 06:31:30.873172 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:30.873174 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:30.873370 (+ 196us) server_negotiation.cc:407] Connection header received
0806 06:31:30.873452 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:30.873452 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:30.873462 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:30.873489 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:30.873942 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:30.874706 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:30.875823 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:30.876131 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:30.876167 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:30.895494 (+ 19327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:30.895495 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:30.895495 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:30.895502 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:30.897116 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:30.897590 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:30.897591 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:30.897591 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:30.897641 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:30.897831 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:30.897849 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:30.897849 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:30.897978 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:30.898019 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:30.898450 (+ 431us) server_negotiation.cc:299] Negotiation successful
0806 06:31:30.898537 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 06:31:30.898681 16142 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } } peers { permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 34313 } attrs { promote: false } } peers { permanent_uuid: "cbb47ccf5ac7455dbb8898888f83dbf3" member_type: NON_VOTER last_known_addr { host: "127.7.43.124" port: 45905 } attrs { promote: true } }
I20260806 06:31:30.899184 17288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:30.872734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.124:45905 (local address 127.0.0.1:58716)
0806 06:31:30.873311 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:30.873320 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:30.873389 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:30.873531 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:30.873532 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:30.873791 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:30.873852 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:30.873854 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:30.874769 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:30.874770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:30.875734 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:30.875737 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:30.876195 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:30.876195 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:30.876287 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:30.894983 (+ 18696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:30.894987 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:30.895421 (+ 434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:30.897586 (+ 2165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:30.897587 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:30.897588 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:30.897765 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:30.898366 (+ 601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:30.898367 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:30.898367 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:30.898428 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:30.898841 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:30.898842 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:30.898884 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 06:31:30.898954 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 06:31:30.899052 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"mutex_wait_us":34,"thread_start_us":225,"threads_started":1}
I20260806 06:31:30.899262 17259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:30.872890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58716 (local address 127.7.43.124:45905)
0806 06:31:30.872923 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:30.872924 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:30.873369 (+ 445us) server_negotiation.cc:407] Connection header received
0806 06:31:30.873449 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:30.873450 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:30.873457 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:30.873482 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:30.873969 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:30.874685 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:30.875821 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:30.876097 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:30.876132 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:30.895492 (+ 19360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:30.895493 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:30.895493 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:30.895498 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:30.897522 (+ 2024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:30.898002 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:30.898003 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:30.898003 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:30.898055 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:30.898487 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:30.898488 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:30.898488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:30.898591 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:30.898641 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:30.898987 (+ 346us) server_negotiation.cc:299] Negotiation successful
0806 06:31:30.899113 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":17}
I20260806 06:31:30.899309 17084 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023 [term 1 FOLLOWER]: Refusing update from remote peer ee7ac2b4afd849459e289fe5dba19836: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260806 06:31:30.900005 17140 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 34313 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 06:31:30.900051 17215 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:30.900754 17141 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER cbb47ccf5ac7455dbb8898888f83dbf3 (127.7.43.124) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } } peers { permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 34313 } attrs { promote: false } } peers { permanent_uuid: "cbb47ccf5ac7455dbb8898888f83dbf3" member_type: NON_VOTER last_known_addr { host: "127.7.43.124" port: 45905 } attrs { promote: true } } }
I20260806 06:31:30.901031 17287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:30.873187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.125:34313 (local address 127.0.0.1:40334)
0806 06:31:30.873311 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:30.873320 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:30.873389 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:30.873542 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:30.873543 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:30.873794 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:30.873855 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:30.873857 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:30.874807 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:30.874808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:30.875734 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:30.875737 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:30.876282 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:30.876283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:30.876331 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:30.894909 (+ 18578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:30.894914 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:30.895376 (+ 462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:30.897205 (+ 1829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:30.897206 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:30.897206 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:30.897505 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:30.897711 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:30.897712 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:30.897712 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:30.897764 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:30.898234 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:30.898234 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:30.898286 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 06:31:30.899835 (+ 1549us) client_negotiation.cc:240] Negotiation successful
0806 06:31:30.900932 (+ 1097us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"mutex_wait_us":20,"thread_start_us":64,"threads_started":1}
I20260806 06:31:30.901906 17084 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER cbb47ccf5ac7455dbb8898888f83dbf3 (127.7.43.124) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } } peers { permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 34313 } attrs { promote: false } } peers { permanent_uuid: "cbb47ccf5ac7455dbb8898888f83dbf3" member_type: NON_VOTER last_known_addr { host: "127.7.43.124" port: 45905 } attrs { promote: true } } }
I20260806 06:31:30.915416 17141 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.7.43.124:45905
I20260806 06:31:30.915437 17215 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3 [term 1 FOLLOWER]: Refusing update from remote peer ee7ac2b4afd849459e289fe5dba19836: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260806 06:31:30.915493 17141 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ee7ac2b4afd849459e289fe5dba19836" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } } peers { permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 34313 } attrs { promote: false } } peers { permanent_uuid: "cbb47ccf5ac7455dbb8898888f83dbf3" member_type: NON_VOTER last_known_addr { host: "127.7.43.124" port: 45905 } attrs { promote: true } } }
I20260806 06:31:30.915551 17141 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836 [sys.catalog]: This master's current role is: LEADER
I20260806 06:31:30.915658 17141 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ee7ac2b4afd849459e289fe5dba19836" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } } peers { permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 34313 } attrs { promote: false } } peers { permanent_uuid: "cbb47ccf5ac7455dbb8898888f83dbf3" member_type: NON_VOTER last_known_addr { host: "127.7.43.124" port: 45905 } attrs { promote: true } } }
I20260806 06:31:30.915695 17141 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836 [sys.catalog]: This master's current role is: LEADER
I20260806 06:31:30.915715 17141 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cbb47ccf5ac7455dbb8898888f83dbf3" member_type: NON_VOTER last_known_addr { host: "127.7.43.124" port: 45905 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:31:30.916129 17152 master.cc:537] Master@127.7.43.124:45905 shutting down...
I20260806 06:31:30.916698 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 16238
I20260806 06:31:30.922576 17292 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ee7ac2b4afd849459e289fe5dba19836" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } } peers { permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 34313 } attrs { promote: false } } peers { permanent_uuid: "cbb47ccf5ac7455dbb8898888f83dbf3" member_type: NON_VOTER last_known_addr { host: "127.7.43.124" port: 45905 } attrs { promote: true } } }
I20260806 06:31:30.922678 17292 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 06:31:30.922804 16119 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260806 06:31:30.923213 17152 raft_consensus.cc:2229] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260806 06:31:30.923224 16119 consensus_peers.cc:487] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836 -> Peer cbb47ccf5ac7455dbb8898888f83dbf3 (127.7.43.124:45905): Couldn't send request to peer cbb47ccf5ac7455dbb8898888f83dbf3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 06:31:30.923305 17152 raft_consensus.cc:2258] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:31:30.923362 17152 tablet_replica.cc:330] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3: stopping tablet replica
I20260806 06:31:30.927068 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.65:39903
--local_ip_for_outbound_sockets=127.7.43.65
--tserver_master_addrs=127.7.43.126:36787
--webserver_port=45349
--webserver_interface=127.7.43.65
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/oryx_127.7.43.65.keytab
--principal=oryx/127.7.43.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:37673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.43.126:36787,127.7.43.125:34313
--tserver_master_addrs=127.7.43.126:36787,127.7.43.125:34313,127.7.43.124:45905 with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:31:30.936007 17152 master.cc:552] Master@127.7.43.124:45905 shutdown complete.
I20260806 06:31:30.936048 17152 master_runner.cc:305] Clearing existing system tablet
I20260806 06:31:30.936172 17152 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:30.936215 17152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:30.936786 17152 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:31:30.937224 17297 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:30.937353 17152 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:30.937462 17152 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:30.937496 17152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-2/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-2/wal
uuid: "cbb47ccf5ac7455dbb8898888f83dbf3"
format_stamp: "Formatted at 2026-08-06 06:31:30 on dist-test-slave-6zbq"
server_key: "7a3b66a886cf07f2f4004d5c0bcd0c21"
server_key_iv: "c121b16cc1e7a800325d4a98c4397d4f"
server_key_version: "encryptionkey@0"
I20260806 06:31:30.938181 17152 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:31:31.018846 17152 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 06:31:31.018941 17152 log.cc:1196] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260806 06:31:31.019125 17152 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3: Deleting consensus metadata
W20260806 06:31:31.042303 17294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:31.042529 17294 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:31.042553 17294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:31.043695 17294 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:31.043736 17294 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:31.043762 17294 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.65
I20260806 06:31:31.045292 17294 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:37673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/oryx_127.7.43.65.keytab
--principal=oryx/127.7.43.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.65:39903
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.43.65
--webserver_port=45349
--tserver_master_addrs=127.7.43.126:36787,127.7.43.125:34313,127.7.43.124:45905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17294
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.65
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:31.045511 17294 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:31.045719 17294 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:31.050586 17306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:31.050613 17294 server_base.cc:1034] running on GCE node
W20260806 06:31:31.050590 17309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:31.050616 17305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:31.050992 17294 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:31.051218 17294 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:31.052347 17294 hybrid_clock.cc:648] HybridClock initialized: now 1785997891052336 us; error 32 us; skew 500 ppm
Aug 06 06:31:31 dist-test-slave-6zbq krb5kdc[16073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997891, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.43.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:31.054679 17294 init.cc:377] Logged in from keytab as oryx/127.7.43.65@KRBTEST.COM (short username oryx)
I20260806 06:31:31.055060 17294 webserver.cc:460] Webserver started at http://127.7.43.65:45349/ using document root <none> and password file <none>
I20260806 06:31:31.055255 17294 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:31.055325 17294 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:31.056519 17294 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:31:31.056994 17318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:31.057140 17294 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:31.057219 17294 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-0/wal
uuid: "17fc103e74544c2a98d06b27aabb9435"
format_stamp: "Formatted at 2026-08-06 06:31:29 on dist-test-slave-6zbq"
server_key: "9d6717dbb8f470ce6b7d0f2361149673"
server_key_iv: "b55d4c3090124ce71c5a80b159a7f2b9"
server_key_version: "encryptionkey@0"
I20260806 06:31:31.057503 17294 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:31.065021 17152 master_runner.cc:315] Copying system tablet from 127.7.43.126:36787
I20260806 06:31:31.065583 17152 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3: tablet copy: Beginning tablet copy session from remote peer at address 127.7.43.126:36787
I20260806 06:31:31.071310 17331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:31.065830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:36787 (local address 127.0.0.1:45668)
0806 06:31:31.065941 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:31.065950 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:31.066006 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:31.066149 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:31.066150 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:31.066293 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:31.066333 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:31.066334 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.067243 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.067243 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:31.067834 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:31.067837 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.068255 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.068256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:31.068298 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:31.068713 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:31.068717 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:31.069009 (+ 292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:31.070498 (+ 1489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:31.070499 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:31.070500 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:31.070668 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:31.070836 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:31.070837 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:31.070837 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:31.070882 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:31.071091 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:31.071091 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:31.071127 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 06:31:31.071171 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 06:31:31.071246 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":61,"threads_started":1}
I20260806 06:31:31.071414 17253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:31.065909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45668 (local address 127.7.43.126:36787)
0806 06:31:31.065957 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:31.065959 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:31.065993 (+ 34us) server_negotiation.cc:407] Connection header received
0806 06:31:31.066060 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:31.066061 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:31.066071 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:31.066134 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:31.066406 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.067178 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.067892 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.068194 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.068254 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:31.069082 (+ 828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:31.069083 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:31.069084 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:31.069090 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:31.070432 (+ 1342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:31.070727 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:31.070733 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:31.070733 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:31.070790 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:31.070929 (+ 139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:31.070929 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:31.070930 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:31.071042 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:31.071091 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:31.071240 (+ 149us) server_negotiation.cc:299] Negotiation successful
0806 06:31:31.071319 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 06:31:31.071593 16172 tablet_copy_service.cc:140] P ee7ac2b4afd849459e289fe5dba19836: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer cbb47ccf5ac7455dbb8898888f83dbf3 ({username='oryx', principal='oryx/127.7.43.124@KRBTEST.COM'} at 127.0.0.1:45668)
I20260806 06:31:31.071672 16172 tablet_copy_service.cc:161] P ee7ac2b4afd849459e289fe5dba19836: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer cbb47ccf5ac7455dbb8898888f83dbf3 at {username='oryx', principal='oryx/127.7.43.124@KRBTEST.COM'} at 127.0.0.1:45668: session id = cbb47ccf5ac7455dbb8898888f83dbf3-00000000000000000000000000000000
I20260806 06:31:31.072085 16172 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 06:31:31.072489 17152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 06:31:31.073508 17152 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3: tablet copy: Starting download of 0 data blocks...
I20260806 06:31:31.073635 17152 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3: tablet copy: Starting download of 1 WAL segments...
I20260806 06:31:31.073827 17294 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:31.074270 17152 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 06:31:31.076000 17294 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:31.076098 17294 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:31.076261 17294 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:31:31.076486 17294 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:31:31.076514 17294 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:31.076536 17294 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:31:31.076552 17294 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:31.082595 17294 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.65:39903
I20260806 06:31:31.082697 17506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.65:39903 every 8 connection(s)
I20260806 06:31:31.082971 17294 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
I20260806 06:31:31.084645 17515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:31.083604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.124:45905 (local address 127.7.43.65:34721)
0806 06:31:31.083773 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:31.083859 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.124:45905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":107,"threads_started":1}
W20260806 06:31:31.084884 17508 heartbeater.cc:644] Failed to heartbeat to 127.7.43.124:45905 (0 consecutive failures): Network error: Failed to ping master at 127.7.43.124:45905: Client connection negotiation failed: client connection to 127.7.43.124:45905: connect: Connection refused (error 111)
I20260806 06:31:31.085237 17515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:31.085039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.124:45905 (local address 127.7.43.65:41323)
0806 06:31:31.085185 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:31.085211 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.124:45905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":79,"threads_started":1}
I20260806 06:31:31.085532 17520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:31.085405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.124:45905 (local address 127.7.43.65:57081)
0806 06:31:31.085459 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:31.085495 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.124:45905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260806 06:31:31.085700 17508 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 06 06:31:31 dist-test-slave-6zbq krb5kdc[16073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997891, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.43.65@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM
Aug 06 06:31:31 dist-test-slave-6zbq krb5kdc[16073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997891, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.43.65@KRBTEST.COM for oryx/127.7.43.125@KRBTEST.COM
I20260806 06:31:31.091771 17253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:31.083639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:53833 (local address 127.7.43.126:36787)
0806 06:31:31.083685 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:31.083687 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:31.083885 (+ 198us) server_negotiation.cc:407] Connection header received
0806 06:31:31.084393 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:31.084394 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:31.084405 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:31.084452 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:31.085201 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.085872 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.086887 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.087083 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.087144 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:31.089095 (+ 1951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:31.089096 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:31.089097 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:31.089103 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:31.090580 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:31.090920 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:31.090922 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:31.090922 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:31.090976 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:31.091190 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:31.091191 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:31.091192 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:31.091296 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:31.091351 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:31.091609 (+ 258us) server_negotiation.cc:299] Negotiation successful
0806 06:31:31.091683 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 06:31:31.091781 17149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:31.083968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:48605 (local address 127.7.43.125:34313)
0806 06:31:31.084010 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:31.084013 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:31.084063 (+ 50us) server_negotiation.cc:407] Connection header received
0806 06:31:31.084435 (+ 372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:31.084436 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:31.084449 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:31.084482 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:31.085201 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.085810 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.086893 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.087089 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.087145 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:31.089492 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:31.089493 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:31.089493 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:31.089500 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:31.090800 (+ 1300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:31.091097 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:31.091098 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:31.091099 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:31.091150 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:31.091309 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:31.091310 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:31.091311 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:31.091418 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:31.091472 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:31.091620 (+ 148us) server_negotiation.cc:299] Negotiation successful
0806 06:31:31.091696 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 06:31:31.091928 17518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:31.083891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.125:34313 (local address 127.7.43.65:48605)
0806 06:31:31.084002 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:31.084012 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:31.084319 (+ 307us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:31.084580 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:31.084581 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:31.084829 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:31.085093 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:31.085101 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.085917 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.085918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:31.086817 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:31.086820 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.087158 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.087159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:31.087271 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:31.087778 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:31.087784 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:31.089383 (+ 1599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:31.090878 (+ 1495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:31.090879 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:31.090880 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:31.091031 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:31.091203 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:31.091203 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:31.091204 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:31.091260 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:31.091472 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:31.091473 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:31.091527 (+ 54us) client_negotiation.cc:769] Sending connection context
0806 06:31:31.091591 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 06:31:31.091755 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"mutex_wait_us":57,"thread_start_us":67,"threads_started":1}
I20260806 06:31:31.091934 17516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:31.083604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:36787 (local address 127.7.43.65:53833)
0806 06:31:31.083779 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:31.083815 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:31.084303 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:31.084512 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:31.084515 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:31.084827 (+ 312us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:31.085092 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:31.085097 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.085949 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.085949 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:31.086813 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:31.086816 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.087152 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.087153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:31.087270 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:31.087716 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:31.087724 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:31.089014 (+ 1290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:31.090653 (+ 1639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:31.090660 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:31.090662 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:31.090856 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:31.091028 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:31.091029 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:31.091029 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:31.091135 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:31.091349 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:31.091356 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:31.091495 (+ 139us) client_negotiation.cc:769] Sending connection context
0806 06:31:31.091581 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 06:31:31.091826 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":119,"thread_start_us":88,"threads_started":1}
I20260806 06:31:31.092409 17511 heartbeater.cc:347] Connected to a master server at 127.7.43.125:34313
I20260806 06:31:31.092429 17510 heartbeater.cc:347] Connected to a master server at 127.7.43.126:36787
I20260806 06:31:31.092481 17511 heartbeater.cc:464] Registering TS with master...
I20260806 06:31:31.092484 17510 heartbeater.cc:464] Registering TS with master...
I20260806 06:31:31.092658 17510 heartbeater.cc:511] Master 127.7.43.126:36787 requested a full tablet report, sending...
I20260806 06:31:31.092662 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 17294
I20260806 06:31:31.092664 17511 heartbeater.cc:511] Master 127.7.43.125:34313 requested a full tablet report, sending...
I20260806 06:31:31.092815 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 16436
I20260806 06:31:31.093137 17066 ts_manager.cc:194] Registered new tserver with Master: 17fc103e74544c2a98d06b27aabb9435 (127.7.43.65:39903)
I20260806 06:31:31.094520 16142 ts_manager.cc:194] Re-registered known tserver with Master: 17fc103e74544c2a98d06b27aabb9435 (127.7.43.65:39903)
I20260806 06:31:31.094895 16142 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.43.65@KRBTEST.COM'} at 127.7.43.65:53833
I20260806 06:31:31.112470 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.66:44525
--local_ip_for_outbound_sockets=127.7.43.66
--tserver_master_addrs=127.7.43.126:36787
--webserver_port=34127
--webserver_interface=127.7.43.66
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/oryx_127.7.43.66.keytab
--principal=oryx/127.7.43.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:37673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.43.126:36787,127.7.43.125:34313
--tserver_master_addrs=127.7.43.126:36787,127.7.43.125:34313,127.7.43.124:45905 with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:31:31.114738 16172 tablet_copy_service.cc:342] P ee7ac2b4afd849459e289fe5dba19836: Request end of tablet copy session cbb47ccf5ac7455dbb8898888f83dbf3-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.7.43.124@KRBTEST.COM'} at 127.0.0.1:45668
I20260806 06:31:31.114816 16172 tablet_copy_service.cc:433] P ee7ac2b4afd849459e289fe5dba19836: ending tablet copy session cbb47ccf5ac7455dbb8898888f83dbf3-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer cbb47ccf5ac7455dbb8898888f83dbf3
I20260806 06:31:31.117587 17152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:31.118574 17522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:31.118746 17152 server_base.cc:1034] running on GCE node
W20260806 06:31:31.118853 17523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:31.121351 17529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:31.121472 17152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:31.121604 17152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:31.122720 17152 hybrid_clock.cc:648] HybridClock initialized: now 1785997891122712 us; error 22 us; skew 500 ppm
Aug 06 06:31:31 dist-test-slave-6zbq krb5kdc[16073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997891, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.43.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:31.124172 17152 init.cc:377] Logged in from keytab as oryx/127.7.43.124@KRBTEST.COM (short username oryx)
I20260806 06:31:31.124445 17152 webserver.cc:460] Webserver started at http://127.7.43.124:43015/ using document root <none> and password file <none>
I20260806 06:31:31.124557 17152 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:31.124593 17152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:31.125280 17152 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:31:31.125777 17543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:31.125898 17152 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:31:31.125938 17152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-2/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-2/wal
uuid: "cbb47ccf5ac7455dbb8898888f83dbf3"
format_stamp: "Formatted at 2026-08-06 06:31:30 on dist-test-slave-6zbq"
server_key: "7a3b66a886cf07f2f4004d5c0bcd0c21"
server_key_iv: "c121b16cc1e7a800325d4a98c4397d4f"
server_key_version: "encryptionkey@0"
I20260806 06:31:31.125993 17152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:31.134653 17152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:31.135927 17152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:31.138744 17152 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.124:45905
I20260806 06:31:31.139529 17635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.124:45905 every 8 connection(s)
W20260806 06:31:31.197799 17521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:31.197965 17521 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:31.197994 17521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:31.199131 17521 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:31.199169 17521 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:31.199195 17521 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.66
I20260806 06:31:31.200629 17521 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:37673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/oryx_127.7.43.66.keytab
--principal=oryx/127.7.43.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.66:44525
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.43.66
--webserver_port=34127
--tserver_master_addrs=127.7.43.126:36787,127.7.43.125:34313,127.7.43.124:45905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17521
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.66
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:31.200826 17521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:31.201030 17521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:31.205514 17643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:31.205503 17642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:31.205504 17647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:31.205740 17521 server_base.cc:1034] running on GCE node
I20260806 06:31:31.205932 17521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:31.206157 17521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:31.207297 17521 hybrid_clock.cc:648] HybridClock initialized: now 1785997891207271 us; error 39 us; skew 500 ppm
Aug 06 06:31:31 dist-test-slave-6zbq krb5kdc[16073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997891, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.43.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:31.209576 17521 init.cc:377] Logged in from keytab as oryx/127.7.43.66@KRBTEST.COM (short username oryx)
I20260806 06:31:31.209954 17521 webserver.cc:460] Webserver started at http://127.7.43.66:34127/ using document root <none> and password file <none>
I20260806 06:31:31.210156 17521 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:31.210217 17521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:31.211483 17521 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:31:31.211969 17658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:31.212106 17521 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:31.212183 17521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-1/wal
uuid: "40b11f4fbad9437eb9f11d2da5dd0d8b"
format_stamp: "Formatted at 2026-08-06 06:31:29 on dist-test-slave-6zbq"
server_key: "5606a74422eec41a4a26837d7cb30bb5"
server_key_iv: "c7f21048e61f909184dbcda8f95756ad"
server_key_version: "encryptionkey@0"
I20260806 06:31:31.212474 17521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:31.221467 17152 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/master-2/data/info.pb
I20260806 06:31:31.222429 17664 sys_catalog.cc:263] Verifying existing consensus state
I20260806 06:31:31.222978 17664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3: Bootstrap starting.
I20260806 06:31:31.225610 17664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:31:31.225826 17664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3: Bootstrap complete.
I20260806 06:31:31.225997 17664 raft_consensus.cc:348] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } } peers { permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 34313 } attrs { promote: false } } peers { permanent_uuid: "cbb47ccf5ac7455dbb8898888f83dbf3" member_type: NON_VOTER last_known_addr { host: "127.7.43.124" port: 45905 } attrs { promote: true } }
I20260806 06:31:31.226076 17664 raft_consensus.cc:729] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cbb47ccf5ac7455dbb8898888f83dbf3, State: Initialized, Role: LEARNER
I20260806 06:31:31.226106 17664 consensus_queue.cc:260] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } } peers { permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 34313 } attrs { promote: false } } peers { permanent_uuid: "cbb47ccf5ac7455dbb8898888f83dbf3" member_type: NON_VOTER last_known_addr { host: "127.7.43.124" port: 45905 } attrs { promote: true } }
I20260806 06:31:31.226264 17664 sys_catalog.cc:564] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:31:31.226271 17667 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } } peers { permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 34313 } attrs { promote: false } } peers { permanent_uuid: "cbb47ccf5ac7455dbb8898888f83dbf3" member_type: NON_VOTER last_known_addr { host: "127.7.43.124" port: 45905 } attrs { promote: true } } }
I20260806 06:31:31.226503 17667 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3 [sys.catalog]: This master's current role is: LEARNER
I20260806 06:31:31.227562 17682 catalog_manager.cc:1259] Loaded cluster ID: f88ac3451d944ee6994511cbff6c6a75
I20260806 06:31:31.227622 17682 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3: loading cluster ID for follower catalog manager: success
I20260806 06:31:31.228567 17682 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3: acquiring CA information for follower catalog manager: success
I20260806 06:31:31.228811 17682 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 06:31:31.238188 17521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:31.239920 17521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:31.240046 17521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:31.240237 17521 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:31:31.240509 17521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:31:31.240542 17521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:31.240569 17521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:31:31.240594 17521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:31.246526 17521 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.66:44525
I20260806 06:31:31.246584 17838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.66:44525 every 8 connection(s)
I20260806 06:31:31.246870 17521 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
I20260806 06:31:31.248700 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 17521
I20260806 06:31:31.248874 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 16634
Aug 06 06:31:31 dist-test-slave-6zbq krb5kdc[16073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997891, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.43.66@KRBTEST.COM for oryx/127.7.43.124@KRBTEST.COM
Aug 06 06:31:31 dist-test-slave-6zbq krb5kdc[16073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997891, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.43.66@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM
Aug 06 06:31:31 dist-test-slave-6zbq krb5kdc[16073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997891, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.43.66@KRBTEST.COM for oryx/127.7.43.125@KRBTEST.COM
I20260806 06:31:31.257426 17848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:31.247553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:54013 (local address 127.7.43.124:45905)
0806 06:31:31.247980 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:31.247982 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:31.247992 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:31:31.248486 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:31.248488 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:31.248498 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:31.248552 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:31.249384 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.250202 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.251410 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.251654 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.251700 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:31.254154 (+ 2454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:31.254155 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:31.254156 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:31.254163 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:31.256107 (+ 1944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:31.256512 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:31.256513 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:31.256514 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:31.256573 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:31.256796 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:31.256797 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:31.256798 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:31.256911 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:31.257023 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:31.257246 (+ 223us) server_negotiation.cc:299] Negotiation successful
0806 06:31:31.257331 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":58,"threads_started":1}
I20260806 06:31:31.257591 17851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:31.247530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.124:45905 (local address 127.7.43.66:54013)
0806 06:31:31.247706 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:31.247742 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:31.248352 (+ 610us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:31.248591 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:31.248591 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:31.249042 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:31.249291 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:31.249296 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.250290 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.250291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:31.251321 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:31.251323 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.251754 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.251755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:31.251868 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:31.252334 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:31.252343 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:31.254034 (+ 1691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:31.256181 (+ 2147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:31.256189 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:31.256191 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:31.256442 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:31.256630 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:31.256631 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:31.256631 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:31.256740 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:31.256966 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:31.256973 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:31.257125 (+ 152us) client_negotiation.cc:769] Sending connection context
0806 06:31:31.257217 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 06:31:31.257370 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":4,"thread_start_us":112,"threads_started":1}
I20260806 06:31:31.258848 17840 heartbeater.cc:347] Connected to a master server at 127.7.43.124:45905
I20260806 06:31:31.259163 17840 heartbeater.cc:464] Registering TS with master...
I20260806 06:31:31.259368 17840 heartbeater.cc:511] Master 127.7.43.124:45905 requested a full tablet report, sending...
I20260806 06:31:31.259879 17565 ts_manager.cc:194] Registered new tserver with Master: 40b11f4fbad9437eb9f11d2da5dd0d8b (127.7.43.66:44525)
I20260806 06:31:31.260789 17149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:31.247994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:58841 (local address 127.7.43.125:34313)
0806 06:31:31.248036 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:31.248038 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:31.248196 (+ 158us) server_negotiation.cc:407] Connection header received
0806 06:31:31.248477 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:31.248478 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:31.248492 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:31.248523 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:31.252816 (+ 4293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.253532 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.254530 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.254832 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.254874 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:31.258205 (+ 3331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:31.258207 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:31.258207 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:31.258215 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:31.259674 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:31.260038 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:31.260040 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:31.260040 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:31.260100 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:31.260304 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:31.260305 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:31.260305 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:31.260437 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:31.260471 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:31.260639 (+ 168us) server_negotiation.cc:299] Negotiation successful
0806 06:31:31.260704 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 06:31:31.260807 17852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:31.247928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.125:34313 (local address 127.7.43.66:58841)
0806 06:31:31.248134 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:31.248142 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:31.248365 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:31.248589 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:31.248589 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:31.252646 (+ 4057us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:31.252709 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:31.252711 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.253648 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.253649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:31.254437 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:31.254440 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.254956 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.254957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:31.255025 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:31.255334 (+ 309us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:31.255339 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:31.258105 (+ 2766us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:31.259754 (+ 1649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:31.259755 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:31.259755 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:31.259951 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:31.260175 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:31.260176 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:31.260176 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:31.260238 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:31.260507 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:31.260508 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:31.260540 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 06:31:31.260600 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 06:31:31.260694 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":155,"threads_started":1}
I20260806 06:31:31.261138 17842 heartbeater.cc:347] Connected to a master server at 127.7.43.125:34313
I20260806 06:31:31.261183 17842 heartbeater.cc:464] Registering TS with master...
I20260806 06:31:31.261238 17842 heartbeater.cc:511] Master 127.7.43.125:34313 requested a full tablet report, sending...
I20260806 06:31:31.261566 17066 ts_manager.cc:194] Registered new tserver with Master: 40b11f4fbad9437eb9f11d2da5dd0d8b (127.7.43.66:44525)
I20260806 06:31:31.269007 17253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:31.247575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:34179 (local address 127.7.43.126:36787)
0806 06:31:31.247620 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:31.247623 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:31.248066 (+ 443us) server_negotiation.cc:407] Connection header received
0806 06:31:31.248436 (+ 370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:31.248438 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:31.248450 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:31.248495 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:31.251688 (+ 3193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.252610 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.253813 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.254173 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.254246 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:31.257454 (+ 3208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:31.257456 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:31.257456 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:31.257465 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:31.267785 (+ 10320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:31.268221 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:31.268223 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:31.268223 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:31.268289 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:31.268510 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:31.268511 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:31.268511 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:31.268646 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:31.268712 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:31.268859 (+ 147us) server_negotiation.cc:299] Negotiation successful
0806 06:31:31.268938 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:31:31.269237 17849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:31.247530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:36787 (local address 127.7.43.66:34179)
0806 06:31:31.247703 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:31.247742 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:31.248353 (+ 611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:31.248574 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:31.248578 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:31.251478 (+ 2900us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:31.251576 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:31.251579 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.252715 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.252716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:31.253701 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:31.253705 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.254323 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.254325 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:31.254428 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:31.254872 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:31.254878 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:31.257321 (+ 2443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:31.267869 (+ 10548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:31.267870 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:31.267870 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:31.268123 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:31.268377 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:31.268378 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:31.268379 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:31.268450 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:31.268711 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:31.268712 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:31.268756 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 06:31:31.268827 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 06:31:31.268928 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":103,"threads_started":1}
I20260806 06:31:31.269578 17841 heartbeater.cc:347] Connected to a master server at 127.7.43.126:36787
I20260806 06:31:31.269630 17841 heartbeater.cc:464] Registering TS with master...
I20260806 06:31:31.269685 17841 heartbeater.cc:511] Master 127.7.43.126:36787 requested a full tablet report, sending...
I20260806 06:31:31.269908 16142 ts_manager.cc:194] Re-registered known tserver with Master: 40b11f4fbad9437eb9f11d2da5dd0d8b (127.7.43.66:44525)
I20260806 06:31:31.270471 16142 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.43.66@KRBTEST.COM'} at 127.7.43.66:34179
I20260806 06:31:31.272812 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.67:41259
--local_ip_for_outbound_sockets=127.7.43.67
--tserver_master_addrs=127.7.43.126:36787
--webserver_port=32991
--webserver_interface=127.7.43.67
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/oryx_127.7.43.67.keytab
--principal=oryx/127.7.43.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:37673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.43.126:36787,127.7.43.125:34313
--tserver_master_addrs=127.7.43.126:36787,127.7.43.125:34313,127.7.43.124:45905 with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:31:31.355199 17853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:31.355374 17853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:31.355396 17853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:31.356452 17853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:31.356483 17853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:31.356508 17853 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.67
I20260806 06:31:31.357874 17853 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:37673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/krb5kdc/oryx_127.7.43.67.keytab
--principal=oryx/127.7.43.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.67:41259
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.43.67
--webserver_port=32991
--tserver_master_addrs=127.7.43.126:36787,127.7.43.125:34313,127.7.43.124:45905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17853
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.67
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:31.358034 17853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:31.358219 17853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:31.362783 17861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:31.362912 17853 server_base.cc:1034] running on GCE node
W20260806 06:31:31.362856 17860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:31.362985 17864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:31.363164 17853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:31.363411 17853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:31.364542 17853 hybrid_clock.cc:648] HybridClock initialized: now 1785997891364524 us; error 30 us; skew 500 ppm
Aug 06 06:31:31 dist-test-slave-6zbq krb5kdc[16073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997891, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.43.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:31.366837 17853 init.cc:377] Logged in from keytab as oryx/127.7.43.67@KRBTEST.COM (short username oryx)
I20260806 06:31:31.367208 17853 webserver.cc:460] Webserver started at http://127.7.43.67:32991/ using document root <none> and password file <none>
I20260806 06:31:31.367370 17853 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:31.367419 17853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:31.368556 17853 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:31:31.369052 17877 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:31.369186 17853 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:31.369263 17853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-2/wal
uuid: "930504b65be34a37ac7455cda9c9c7b6"
format_stamp: "Formatted at 2026-08-06 06:31:29 on dist-test-slave-6zbq"
server_key: "c3e89baab1c20d8e30d864100246eb6a"
server_key_iv: "b0a6e6ffbb682bd7834eec1bdcd28f84"
server_key_version: "encryptionkey@0"
I20260806 06:31:31.369544 17853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:31.384764 17142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:31.382074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.124:45905 (local address 127.0.0.1:58736)
0806 06:31:31.382125 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:31.382135 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:31.382174 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:31.382332 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:31.382333 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:31.382363 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:31:31.382423 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:31.382424 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.383041 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.383043 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:31.383940 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:31.383944 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.384522 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.384523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:31.384593 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:31.384597 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:31:31.384645 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 06:31:31.384693 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 06:31:31.384763 17848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:31.382147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58736 (local address 127.7.43.124:45905)
0806 06:31:31.382209 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:31.382211 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:31.382219 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:31:31.382247 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:31.382247 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:31.382270 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:31.382322 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:31:31.382483 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.382967 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.384024 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.384443 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.384506 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:31.384578 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:31.384656 (+ 78us) server_negotiation.cc:299] Negotiation successful
0806 06:31:31.384698 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 06:31:31.385128 17583 raft_consensus.cc:1203] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.7] Dedup: 1.7->[]
I20260806 06:31:31.385281 17667 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee7ac2b4afd849459e289fe5dba19836. Latest consensus state: current_term: 1 leader_uuid: "ee7ac2b4afd849459e289fe5dba19836" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } } peers { permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 34313 } attrs { promote: false } } peers { permanent_uuid: "cbb47ccf5ac7455dbb8898888f83dbf3" member_type: NON_VOTER last_known_addr { host: "127.7.43.124" port: 45905 } attrs { promote: true } } }
I20260806 06:31:31.385397 17667 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3 [sys.catalog]: This master's current role is: LEARNER
I20260806 06:31:31.389309 17853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:31.391439 17853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:31.391554 17853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:31.391714 17853 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:31:31.391943 17853 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:31:31.391970 17853 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:31.391993 17853 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:31:31.392009 17853 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:31.397392 17853 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.67:41259
I20260806 06:31:31.397459 18059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.67:41259 every 8 connection(s)
I20260806 06:31:31.397753 17853 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
Aug 06 06:31:31 dist-test-slave-6zbq krb5kdc[16073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997891, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.43.67@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM
Aug 06 06:31:31 dist-test-slave-6zbq krb5kdc[16073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997891, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.43.67@KRBTEST.COM for oryx/127.7.43.125@KRBTEST.COM
Aug 06 06:31:31 dist-test-slave-6zbq krb5kdc[16073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997891, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.43.67@KRBTEST.COM for oryx/127.7.43.124@KRBTEST.COM
I20260806 06:31:31.407554 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 17853
I20260806 06:31:31.408661 17149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:31.398720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:39061 (local address 127.7.43.125:34313)
0806 06:31:31.398762 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:31.398763 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:31.398971 (+ 208us) server_negotiation.cc:407] Connection header received
0806 06:31:31.399719 (+ 748us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:31.399720 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:31.399732 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:31.399790 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:31.400455 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.401210 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.402444 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.402714 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.402769 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:31.405430 (+ 2661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:31.405432 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:31.405432 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:31.405441 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:31.407090 (+ 1649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:31.407836 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:31.407837 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:31.407838 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:31.407888 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:31.408075 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:31.408076 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:31.408076 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:31.408190 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:31.408227 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:31.408492 (+ 265us) server_negotiation.cc:299] Negotiation successful
0806 06:31:31.408571 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:31:31.408660 17253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:31.398597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:34373 (local address 127.7.43.126:36787)
0806 06:31:31.398657 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:31.398659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:31.398865 (+ 206us) server_negotiation.cc:407] Connection header received
0806 06:31:31.399787 (+ 922us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:31.399788 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:31.399798 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:31.399841 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:31.400642 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.401143 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.402258 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.402556 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.402617 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:31.405837 (+ 3220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:31.405839 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:31.405839 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:31.405846 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:31.407428 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:31.407703 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:31.407704 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:31.407704 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:31.407761 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:31.407980 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:31.407982 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:31.407982 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:31.408101 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:31.408142 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:31.408492 (+ 350us) server_negotiation.cc:299] Negotiation successful
0806 06:31:31.408566 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 06:31:31.408668 17848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:31.398404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:45645 (local address 127.7.43.124:45905)
0806 06:31:31.398450 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:31.398452 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:31.398682 (+ 230us) server_negotiation.cc:407] Connection header received
0806 06:31:31.399735 (+ 1053us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:31.399736 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:31.399748 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:31.399774 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:31.400476 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.401211 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.402655 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.402941 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.402983 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:31.405792 (+ 2809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:31.405793 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:31.405794 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:31.405801 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:31.407190 (+ 1389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:31.407621 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:31.407622 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:31.407622 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:31.407683 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:31.407978 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:31.407979 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:31.407979 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:31.408100 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:31.408145 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:31.408508 (+ 363us) server_negotiation.cc:299] Negotiation successful
0806 06:31:31.408586 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 06:31:31.409246 18067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:31.398389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.124:45905 (local address 127.7.43.67:45645)
0806 06:31:31.398570 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:31.398609 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:31.399622 (+ 1013us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:31.399866 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:31.399870 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:31.400176 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:31.400367 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:31.400373 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.401276 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.401277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:31.402568 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:31.402571 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.403053 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.403054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:31.403124 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:31.403425 (+ 301us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:31.403430 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:31.405712 (+ 2282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:31.407267 (+ 1555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:31.407268 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:31.407268 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:31.407528 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:31.407757 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:31.407757 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:31.407758 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:31.407903 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:31.408195 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:31.408196 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:31.408346 (+ 150us) client_negotiation.cc:769] Sending connection context
0806 06:31:31.408779 (+ 433us) client_negotiation.cc:240] Negotiation successful
0806 06:31:31.408991 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":55,"thread_start_us":95,"threads_started":1}
I20260806 06:31:31.409248 18066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:31.398674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.125:34313 (local address 127.7.43.67:39061)
0806 06:31:31.398915 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:31.398922 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:31.399627 (+ 705us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:31.399866 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:31.399870 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:31.400167 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:31.400365 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:31.400370 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.401487 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.401487 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:31.402374 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:31.402378 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.402779 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.402780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:31.402844 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:31.403286 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:31.403291 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:31.405340 (+ 2049us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:31.407179 (+ 1839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:31.407186 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:31.407188 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:31.407759 (+ 571us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:31.407958 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:31.407959 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:31.407959 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:31.408009 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:31.408269 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:31.408270 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:31.408344 (+ 74us) client_negotiation.cc:769] Sending connection context
0806 06:31:31.408819 (+ 475us) client_negotiation.cc:240] Negotiation successful
0806 06:31:31.409080 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"mutex_wait_us":154,"thread_start_us":72,"threads_started":1}
I20260806 06:31:31.410406 18062 heartbeater.cc:347] Connected to a master server at 127.7.43.125:34313
I20260806 06:31:31.410494 18062 heartbeater.cc:464] Registering TS with master...
I20260806 06:31:31.410670 18062 heartbeater.cc:511] Master 127.7.43.125:34313 requested a full tablet report, sending...
I20260806 06:31:31.411088 17066 ts_manager.cc:194] Registered new tserver with Master: 930504b65be34a37ac7455cda9c9c7b6 (127.7.43.67:41259)
I20260806 06:31:31.409654 18069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:31.398389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:36787 (local address 127.7.43.67:34373)
0806 06:31:31.398808 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:31.398816 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:31.399628 (+ 812us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:31.399877 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:31.399877 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:31.400526 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:31.400572 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:31.400574 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.401246 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.401247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:31.402175 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:31.402177 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.402647 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.402649 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:31.402773 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:31.403186 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:31.403195 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:31.405764 (+ 2569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:31.407488 (+ 1724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:31.407488 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:31.407489 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:31.407646 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:31.407828 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:31.407829 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:31.407829 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:31.407908 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:31.408184 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:31.408190 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:31.408344 (+ 154us) client_negotiation.cc:769] Sending connection context
0806 06:31:31.409159 (+ 815us) client_negotiation.cc:240] Negotiation successful
0806 06:31:31.409254 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":385,"mutex_wait_us":67,"thread_start_us":90,"threads_started":1}
I20260806 06:31:31.409926 18060 heartbeater.cc:347] Connected to a master server at 127.7.43.124:45905
I20260806 06:31:31.411406 18060 heartbeater.cc:464] Registering TS with master...
I20260806 06:31:31.411473 18060 heartbeater.cc:511] Master 127.7.43.124:45905 requested a full tablet report, sending...
I20260806 06:31:31.411729 17565 ts_manager.cc:194] Registered new tserver with Master: 930504b65be34a37ac7455cda9c9c7b6 (127.7.43.67:41259)
I20260806 06:31:31.412137 18061 heartbeater.cc:347] Connected to a master server at 127.7.43.126:36787
I20260806 06:31:31.412189 18061 heartbeater.cc:464] Registering TS with master...
I20260806 06:31:31.412245 18061 heartbeater.cc:511] Master 127.7.43.126:36787 requested a full tablet report, sending...
I20260806 06:31:31.412456 16142 ts_manager.cc:194] Re-registered known tserver with Master: 930504b65be34a37ac7455cda9c9c7b6 (127.7.43.67:41259)
I20260806 06:31:31.412925 16142 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.43.67@KRBTEST.COM'} at 127.7.43.67:34373
I20260806 06:31:31.416850 17253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:31.408621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45684 (local address 127.7.43.126:36787)
0806 06:31:31.408730 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:31.408732 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:31.409493 (+ 761us) server_negotiation.cc:407] Connection header received
0806 06:31:31.409580 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:31.409580 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:31.409590 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:31.409618 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:31.410373 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.411089 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.412169 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.412452 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.412487 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:31.414038 (+ 1551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:31.414039 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:31.414040 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:31.414046 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:31.415753 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:31.416125 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:31.416126 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:31.416126 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:31.416179 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:31.416346 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:31.416347 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:31.416347 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:31.416458 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:31.416555 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:31.416695 (+ 140us) server_negotiation.cc:299] Negotiation successful
0806 06:31:31.416771 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":75,"threads_started":1}
I20260806 06:31:31.417512 17848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:31.409224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58746 (local address 127.7.43.124:45905)
0806 06:31:31.409256 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:31.409258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:31.409647 (+ 389us) server_negotiation.cc:407] Connection header received
0806 06:31:31.409747 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:31.409749 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:31.409760 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:31.409815 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:31.410382 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.411092 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.412169 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.412472 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.412505 (+ 33us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:31.414062 (+ 1557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:31.414063 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:31.414063 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:31.414070 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:31.415719 (+ 1649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:31.416209 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:31.416210 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:31.416211 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:31.416267 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:31.417008 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:31.417009 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:31.417009 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:31.417130 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:31.417212 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:31.417368 (+ 156us) server_negotiation.cc:299] Negotiation successful
0806 06:31:31.417443 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 06:31:31.417781 17149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:31.408982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40348 (local address 127.7.43.125:34313)
0806 06:31:31.409009 (+ 27us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:31.409010 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:31.409971 (+ 961us) server_negotiation.cc:407] Connection header received
0806 06:31:31.409989 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:31.409990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:31.409998 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:31.410044 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:31.411070 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.411688 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.413111 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.413387 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.413426 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:31.414802 (+ 1376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:31.414803 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:31.414803 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:31.414810 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:31.416352 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:31.416638 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:31.416639 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:31.416639 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:31.416692 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:31.417236 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:31.417237 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:31.417238 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:31.417361 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:31.417442 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:31.417626 (+ 184us) server_negotiation.cc:299] Negotiation successful
0806 06:31:31.417698 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260806 06:31:31.418478 16142 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45684:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:31:31.419967 16142 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:31:31.420639 17141 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ee7ac2b4afd849459e289fe5dba19836" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } } peers { permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 34313 } attrs { promote: false } } peers { permanent_uuid: "cbb47ccf5ac7455dbb8898888f83dbf3" member_type: NON_VOTER last_known_addr { host: "127.7.43.124" port: 45905 } attrs { promote: true } } }
I20260806 06:31:31.420727 17141 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836 [sys.catalog]: This master's current role is: LEADER
I20260806 06:31:31.421877 17141 raft_consensus.cc:1050] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836: attempting to promote NON_VOTER cbb47ccf5ac7455dbb8898888f83dbf3 to VOTER
I20260806 06:31:31.421998 17141 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } } peers { permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 34313 } attrs { promote: false } } peers { permanent_uuid: "cbb47ccf5ac7455dbb8898888f83dbf3" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 45905 } attrs { promote: false } }
I20260806 06:31:31.422506 17583 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3 [term 1 LEARNER]: Refusing update from remote peer ee7ac2b4afd849459e289fe5dba19836: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260806 06:31:31.422575 17084 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023 [term 1 FOLLOWER]: Refusing update from remote peer ee7ac2b4afd849459e289fe5dba19836: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260806 06:31:31.422704 18088 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cbb47ccf5ac7455dbb8898888f83dbf3" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 45905 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260806 06:31:31.422806 18089 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 34313 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260806 06:31:31.423446 18088 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, cbb47ccf5ac7455dbb8898888f83dbf3 (127.7.43.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } } peers { permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 34313 } attrs { promote: false } } peers { permanent_uuid: "cbb47ccf5ac7455dbb8898888f83dbf3" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 45905 } attrs { promote: false } } }
I20260806 06:31:31.423565 17084 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, cbb47ccf5ac7455dbb8898888f83dbf3 (127.7.43.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } } peers { permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 34313 } attrs { promote: false } } peers { permanent_uuid: "cbb47ccf5ac7455dbb8898888f83dbf3" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 45905 } attrs { promote: false } } }
I20260806 06:31:31.424131 17583 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, cbb47ccf5ac7455dbb8898888f83dbf3 (127.7.43.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } } peers { permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 34313 } attrs { promote: false } } peers { permanent_uuid: "cbb47ccf5ac7455dbb8898888f83dbf3" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 45905 } attrs { promote: false } } }
I20260806 06:31:31.429736 18088 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ee7ac2b4afd849459e289fe5dba19836" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } } peers { permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 34313 } attrs { promote: false } } peers { permanent_uuid: "cbb47ccf5ac7455dbb8898888f83dbf3" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 45905 } attrs { promote: false } } }
I20260806 06:31:31.429816 18088 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836 [sys.catalog]: This master's current role is: LEADER
I20260806 06:31:31.429888 17667 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ee7ac2b4afd849459e289fe5dba19836" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } } peers { permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 34313 } attrs { promote: false } } peers { permanent_uuid: "cbb47ccf5ac7455dbb8898888f83dbf3" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 45905 } attrs { promote: false } } }
I20260806 06:31:31.429955 17667 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:31:31.430107 18098 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ee7ac2b4afd849459e289fe5dba19836" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } } peers { permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 34313 } attrs { promote: false } } peers { permanent_uuid: "cbb47ccf5ac7455dbb8898888f83dbf3" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 45905 } attrs { promote: false } } }
I20260806 06:31:31.430183 18098 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:31:31.430699 18088 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ee7ac2b4afd849459e289fe5dba19836" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } } peers { permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 34313 } attrs { promote: false } } peers { permanent_uuid: "cbb47ccf5ac7455dbb8898888f83dbf3" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 45905 } attrs { promote: false } } }
I20260806 06:31:31.430794 18088 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836 [sys.catalog]: This master's current role is: LEADER
I20260806 06:31:31.430858 18088 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ee7ac2b4afd849459e289fe5dba19836" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } } peers { permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 34313 } attrs { promote: false } } peers { permanent_uuid: "cbb47ccf5ac7455dbb8898888f83dbf3" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 45905 } attrs { promote: false } } }
I20260806 06:31:31.430907 18088 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836 [sys.catalog]: This master's current role is: LEADER
I20260806 06:31:31.434929 18102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:31.431117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55344 (local address 127.7.43.66:44525)
0806 06:31:31.431496 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:31.431500 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:31.431508 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:31:31.431539 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:31.431541 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:31.431553 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:31.431617 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:31:31.431786 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.432896 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.433950 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.434563 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.434639 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:31.434712 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:31.434808 (+ 96us) server_negotiation.cc:299] Negotiation successful
0806 06:31:31.434850 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":64,"threads_started":1}
I20260806 06:31:31.435144 18099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:31.430756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.66:44525 (local address 127.0.0.1:55344)
0806 06:31:31.431196 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:31.431205 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:31.431252 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:31.431663 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:31.431664 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:31.431666 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:31:31.431706 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:31.431708 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.432989 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.432990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:31.433803 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:31.433805 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.434655 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.434656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:31.434710 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:31.434714 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:31:31.435022 (+ 308us) client_negotiation.cc:240] Negotiation successful
0806 06:31:31.435066 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":70,"threads_started":1}
I20260806 06:31:31.435537 18110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:31.431126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40326 (local address 127.7.43.65:39903)
0806 06:31:31.431362 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:31.431367 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:31.431377 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:31:31.432020 (+ 643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:31.432024 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:31.432042 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:31.432113 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:31:31.432285 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.433311 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.434526 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.435210 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.435281 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:31.435338 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:31.435406 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 06:31:31.435442 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":84,"threads_started":1}
I20260806 06:31:31.435573 17142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:31.430570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.67:41259 (local address 127.0.0.1:45130)
0806 06:31:31.430934 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:31.430945 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:31.430987 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:31.431702 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:31.431703 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:31.431705 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:31:31.431756 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:31.431758 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.433194 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.433195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:31.434230 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:31.434232 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.435330 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.435331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:31.435392 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:31.435395 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:31:31.435424 (+ 29us) client_negotiation.cc:240] Negotiation successful
0806 06:31:31.435465 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348}
I20260806 06:31:31.435525 18101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:31.430572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:39903 (local address 127.0.0.1:40326)
0806 06:31:31.431302 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:31.431312 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:31.431367 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:31.432180 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:31.432181 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:31.432183 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:31:31.432219 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:31.432220 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.433398 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.433399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:31.434453 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:31.434456 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.435281 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.435282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:31.435338 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:31.435342 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:31:31.435393 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 06:31:31.435430 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":717,"mutex_wait_us":507,"thread_start_us":65,"threads_started":1}
I20260806 06:31:31.435783 18100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:31.430932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45130 (local address 127.7.43.67:41259)
0806 06:31:31.431291 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:31.431296 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:31.431306 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:31:31.431340 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:31.431342 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:31.431356 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:31.431827 (+ 471us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:31:31.431871 (+ 44us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.433120 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.434323 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.435230 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.435566 (+ 336us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:31.435648 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:31.435678 (+ 30us) server_negotiation.cc:299] Negotiation successful
0806 06:31:31.435716 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":93,"threads_started":1}
I20260806 06:31:31.435894 17745 tablet_service.cc:1467] Processing CreateTablet for tablet 3656116260e94f3ab2daf2b235e0446e (DEFAULT_TABLE table=test-table [id=7f89644ef4154825bcac8b6eb554e0e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:31:31.436290 17745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3656116260e94f3ab2daf2b235e0446e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:31.436368 17955 tablet_service.cc:1467] Processing CreateTablet for tablet 3656116260e94f3ab2daf2b235e0446e (DEFAULT_TABLE table=test-table [id=7f89644ef4154825bcac8b6eb554e0e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:31:31.436676 17955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3656116260e94f3ab2daf2b235e0446e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:31.439883 18111 tablet_bootstrap.cc:492] T 3656116260e94f3ab2daf2b235e0446e P 930504b65be34a37ac7455cda9c9c7b6: Bootstrap starting.
I20260806 06:31:31.439976 18112 tablet_bootstrap.cc:492] T 3656116260e94f3ab2daf2b235e0446e P 40b11f4fbad9437eb9f11d2da5dd0d8b: Bootstrap starting.
I20260806 06:31:31.440515 18111 tablet_bootstrap.cc:654] T 3656116260e94f3ab2daf2b235e0446e P 930504b65be34a37ac7455cda9c9c7b6: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:31.440515 18112 tablet_bootstrap.cc:654] T 3656116260e94f3ab2daf2b235e0446e P 40b11f4fbad9437eb9f11d2da5dd0d8b: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:31.440824 18112 log.cc:824] T 3656116260e94f3ab2daf2b235e0446e P 40b11f4fbad9437eb9f11d2da5dd0d8b: Log is configured to *not* fsync() on all Append() calls
I20260806 06:31:31.440833 18111 log.cc:824] T 3656116260e94f3ab2daf2b235e0446e P 930504b65be34a37ac7455cda9c9c7b6: Log is configured to *not* fsync() on all Append() calls
I20260806 06:31:31.440578 17411 tablet_service.cc:1467] Processing CreateTablet for tablet 3656116260e94f3ab2daf2b235e0446e (DEFAULT_TABLE table=test-table [id=7f89644ef4154825bcac8b6eb554e0e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:31:31.441383 17411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3656116260e94f3ab2daf2b235e0446e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:31.441612 18112 tablet_bootstrap.cc:492] T 3656116260e94f3ab2daf2b235e0446e P 40b11f4fbad9437eb9f11d2da5dd0d8b: No bootstrap required, opened a new log
I20260806 06:31:31.441637 18111 tablet_bootstrap.cc:492] T 3656116260e94f3ab2daf2b235e0446e P 930504b65be34a37ac7455cda9c9c7b6: No bootstrap required, opened a new log
I20260806 06:31:31.441671 18112 ts_tablet_manager.cc:1397] T 3656116260e94f3ab2daf2b235e0446e P 40b11f4fbad9437eb9f11d2da5dd0d8b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:31:31.441694 18111 ts_tablet_manager.cc:1397] T 3656116260e94f3ab2daf2b235e0446e P 930504b65be34a37ac7455cda9c9c7b6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:31:31.443306 18112 raft_consensus.cc:348] T 3656116260e94f3ab2daf2b235e0446e P 40b11f4fbad9437eb9f11d2da5dd0d8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17fc103e74544c2a98d06b27aabb9435" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 39903 } } peers { permanent_uuid: "930504b65be34a37ac7455cda9c9c7b6" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 41259 } } peers { permanent_uuid: "40b11f4fbad9437eb9f11d2da5dd0d8b" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 44525 } }
I20260806 06:31:31.443431 18112 raft_consensus.cc:374] T 3656116260e94f3ab2daf2b235e0446e P 40b11f4fbad9437eb9f11d2da5dd0d8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:31.443462 18112 raft_consensus.cc:729] T 3656116260e94f3ab2daf2b235e0446e P 40b11f4fbad9437eb9f11d2da5dd0d8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40b11f4fbad9437eb9f11d2da5dd0d8b, State: Initialized, Role: FOLLOWER
I20260806 06:31:31.443589 18112 consensus_queue.cc:260] T 3656116260e94f3ab2daf2b235e0446e P 40b11f4fbad9437eb9f11d2da5dd0d8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17fc103e74544c2a98d06b27aabb9435" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 39903 } } peers { permanent_uuid: "930504b65be34a37ac7455cda9c9c7b6" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 41259 } } peers { permanent_uuid: "40b11f4fbad9437eb9f11d2da5dd0d8b" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 44525 } }
I20260806 06:31:31.443825 18112 ts_tablet_manager.cc:1428] T 3656116260e94f3ab2daf2b235e0446e P 40b11f4fbad9437eb9f11d2da5dd0d8b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:31:31.444099 17841 heartbeater.cc:503] Master 127.7.43.126:36787 was elected leader, sending a full tablet report...
I20260806 06:31:31.444294 18111 raft_consensus.cc:348] T 3656116260e94f3ab2daf2b235e0446e P 930504b65be34a37ac7455cda9c9c7b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17fc103e74544c2a98d06b27aabb9435" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 39903 } } peers { permanent_uuid: "930504b65be34a37ac7455cda9c9c7b6" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 41259 } } peers { permanent_uuid: "40b11f4fbad9437eb9f11d2da5dd0d8b" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 44525 } }
I20260806 06:31:31.444455 18111 raft_consensus.cc:374] T 3656116260e94f3ab2daf2b235e0446e P 930504b65be34a37ac7455cda9c9c7b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:31.444487 18111 raft_consensus.cc:729] T 3656116260e94f3ab2daf2b235e0446e P 930504b65be34a37ac7455cda9c9c7b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 930504b65be34a37ac7455cda9c9c7b6, State: Initialized, Role: FOLLOWER
I20260806 06:31:31.444603 18111 consensus_queue.cc:260] T 3656116260e94f3ab2daf2b235e0446e P 930504b65be34a37ac7455cda9c9c7b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17fc103e74544c2a98d06b27aabb9435" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 39903 } } peers { permanent_uuid: "930504b65be34a37ac7455cda9c9c7b6" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 41259 } } peers { permanent_uuid: "40b11f4fbad9437eb9f11d2da5dd0d8b" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 44525 } }
I20260806 06:31:31.444948 18111 ts_tablet_manager.cc:1428] T 3656116260e94f3ab2daf2b235e0446e P 930504b65be34a37ac7455cda9c9c7b6: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 06:31:31.445665 18061 heartbeater.cc:503] Master 127.7.43.126:36787 was elected leader, sending a full tablet report...
I20260806 06:31:31.445990 18122 tablet_bootstrap.cc:492] T 3656116260e94f3ab2daf2b235e0446e P 17fc103e74544c2a98d06b27aabb9435: Bootstrap starting.
I20260806 06:31:31.446650 18122 tablet_bootstrap.cc:654] T 3656116260e94f3ab2daf2b235e0446e P 17fc103e74544c2a98d06b27aabb9435: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:31.446910 18122 log.cc:824] T 3656116260e94f3ab2daf2b235e0446e P 17fc103e74544c2a98d06b27aabb9435: Log is configured to *not* fsync() on all Append() calls
I20260806 06:31:31.447527 18122 tablet_bootstrap.cc:492] T 3656116260e94f3ab2daf2b235e0446e P 17fc103e74544c2a98d06b27aabb9435: No bootstrap required, opened a new log
I20260806 06:31:31.447576 18122 ts_tablet_manager.cc:1397] T 3656116260e94f3ab2daf2b235e0446e P 17fc103e74544c2a98d06b27aabb9435: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 06:31:31.449066 18122 raft_consensus.cc:348] T 3656116260e94f3ab2daf2b235e0446e P 17fc103e74544c2a98d06b27aabb9435 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17fc103e74544c2a98d06b27aabb9435" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 39903 } } peers { permanent_uuid: "930504b65be34a37ac7455cda9c9c7b6" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 41259 } } peers { permanent_uuid: "40b11f4fbad9437eb9f11d2da5dd0d8b" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 44525 } }
I20260806 06:31:31.449180 18122 raft_consensus.cc:374] T 3656116260e94f3ab2daf2b235e0446e P 17fc103e74544c2a98d06b27aabb9435 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:31.449200 18122 raft_consensus.cc:729] T 3656116260e94f3ab2daf2b235e0446e P 17fc103e74544c2a98d06b27aabb9435 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17fc103e74544c2a98d06b27aabb9435, State: Initialized, Role: FOLLOWER
I20260806 06:31:31.449301 18122 consensus_queue.cc:260] T 3656116260e94f3ab2daf2b235e0446e P 17fc103e74544c2a98d06b27aabb9435 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17fc103e74544c2a98d06b27aabb9435" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 39903 } } peers { permanent_uuid: "930504b65be34a37ac7455cda9c9c7b6" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 41259 } } peers { permanent_uuid: "40b11f4fbad9437eb9f11d2da5dd0d8b" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 44525 } }
I20260806 06:31:31.449561 18122 ts_tablet_manager.cc:1428] T 3656116260e94f3ab2daf2b235e0446e P 17fc103e74544c2a98d06b27aabb9435: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 06:31:31.449709 17510 heartbeater.cc:503] Master 127.7.43.126:36787 was elected leader, sending a full tablet report...
I20260806 06:31:31.449625 18121 raft_consensus.cc:482] T 3656116260e94f3ab2daf2b235e0446e P 930504b65be34a37ac7455cda9c9c7b6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:31:31.449925 18121 raft_consensus.cc:504] T 3656116260e94f3ab2daf2b235e0446e P 930504b65be34a37ac7455cda9c9c7b6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17fc103e74544c2a98d06b27aabb9435" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 39903 } } peers { permanent_uuid: "930504b65be34a37ac7455cda9c9c7b6" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 41259 } } peers { permanent_uuid: "40b11f4fbad9437eb9f11d2da5dd0d8b" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 44525 } }
I20260806 06:31:31.450604 18121 leader_election.cc:288] T 3656116260e94f3ab2daf2b235e0446e P 930504b65be34a37ac7455cda9c9c7b6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17fc103e74544c2a98d06b27aabb9435 (127.7.43.65:39903), 40b11f4fbad9437eb9f11d2da5dd0d8b (127.7.43.66:44525)
I20260806 06:31:31.452872 17848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:31.450151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:39015 (local address 127.7.43.124:45905)
0806 06:31:31.450193 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:31.450195 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:31.450202 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:31:31.450221 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:31.450222 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:31.450230 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:31.450272 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:31:31.450520 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.451139 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.452239 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.452585 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.452643 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:31.452651 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:31.452785 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 06:31:31.452814 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:31:31.452875 17518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:31.449978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.124:45905 (local address 127.7.43.65:39015)
0806 06:31:31.450027 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:31.450038 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:31.450082 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:31.450317 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:31.450318 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:31.450319 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:31:31.450439 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:31.450441 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.451208 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.451209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:31.452162 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:31.452164 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.452649 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.452651 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:31.452728 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:31.452731 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:31:31.452775 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 06:31:31.452813 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260806 06:31:31.453217 17508 heartbeater.cc:347] Connected to a master server at 127.7.43.124:45905
I20260806 06:31:31.453275 17508 heartbeater.cc:464] Registering TS with master...
I20260806 06:31:31.453320 17508 heartbeater.cc:511] Master 127.7.43.124:45905 requested a full tablet report, sending...
I20260806 06:31:31.453577 17564 ts_manager.cc:194] Registered new tserver with Master: 17fc103e74544c2a98d06b27aabb9435 (127.7.43.65:39903)
I20260806 06:31:31.453804 18066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:31.450730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.66:44525 (local address 127.7.43.67:57139)
0806 06:31:31.450768 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:31.450778 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:31.450810 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:31.450942 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:31.450943 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:31.450944 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:31:31.450999 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:31.451000 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.451773 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.451774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:31.452923 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:31.452926 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.453572 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.453573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:31.453647 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:31.453651 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:31:31.453705 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 06:31:31.453738 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260806 06:31:31.454110 18102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:31.450796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:57139 (local address 127.7.43.66:44525)
0806 06:31:31.450846 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:31.450848 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:31.450855 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:31:31.450874 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:31.450875 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:31.450884 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:31.450937 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:31:31.451071 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.451692 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.452997 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.453509 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.453570 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:31.453580 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:31.454017 (+ 437us) server_negotiation.cc:299] Negotiation successful
0806 06:31:31.454055 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:31:31.454509 17773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3656116260e94f3ab2daf2b235e0446e" candidate_uuid: "930504b65be34a37ac7455cda9c9c7b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40b11f4fbad9437eb9f11d2da5dd0d8b" is_pre_election: true
I20260806 06:31:31.454632 17773 raft_consensus.cc:2454] T 3656116260e94f3ab2daf2b235e0446e P 40b11f4fbad9437eb9f11d2da5dd0d8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 930504b65be34a37ac7455cda9c9c7b6 in term 0.
I20260806 06:31:31.454886 17880 leader_election.cc:302] T 3656116260e94f3ab2daf2b235e0446e P 930504b65be34a37ac7455cda9c9c7b6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40b11f4fbad9437eb9f11d2da5dd0d8b, 930504b65be34a37ac7455cda9c9c7b6; no voters:
I20260806 06:31:31.455027 18121 raft_consensus.cc:2781] T 3656116260e94f3ab2daf2b235e0446e P 930504b65be34a37ac7455cda9c9c7b6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:31:31.455062 18110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:31.451231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:43449 (local address 127.7.43.65:39903)
0806 06:31:31.451284 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:31.451286 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:31.451292 (+ 6us) server_negotiation.cc:407] Connection header received
0806 06:31:31.451310 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:31.451311 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:31.451389 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:31.451441 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:31:31.451571 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.453010 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.454234 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.454765 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.454843 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:31.454852 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:31.454973 (+ 121us) server_negotiation.cc:299] Negotiation successful
0806 06:31:31.455008 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 06:31:31.455086 18121 raft_consensus.cc:482] T 3656116260e94f3ab2daf2b235e0446e P 930504b65be34a37ac7455cda9c9c7b6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:31:31.455096 18069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:31.450515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:39903 (local address 127.7.43.67:43449)
0806 06:31:31.450571 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:31.450582 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:31.450616 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:31.451460 (+ 844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:31.451461 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:31.451463 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:31:31.451515 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:31.451516 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.453091 (+ 1575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.453092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:31.454159 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:31.454162 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.454844 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.454845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:31.454912 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:31.454915 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:31:31.454954 (+ 39us) client_negotiation.cc:240] Negotiation successful
0806 06:31:31.454987 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 06:31:31.455108 18121 raft_consensus.cc:3037] T 3656116260e94f3ab2daf2b235e0446e P 930504b65be34a37ac7455cda9c9c7b6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:31.455574 17442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3656116260e94f3ab2daf2b235e0446e" candidate_uuid: "930504b65be34a37ac7455cda9c9c7b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17fc103e74544c2a98d06b27aabb9435" is_pre_election: true
I20260806 06:31:31.455664 17442 raft_consensus.cc:2454] T 3656116260e94f3ab2daf2b235e0446e P 17fc103e74544c2a98d06b27aabb9435 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 930504b65be34a37ac7455cda9c9c7b6 in term 0.
I20260806 06:31:31.456043 18121 raft_consensus.cc:504] T 3656116260e94f3ab2daf2b235e0446e P 930504b65be34a37ac7455cda9c9c7b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17fc103e74544c2a98d06b27aabb9435" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 39903 } } peers { permanent_uuid: "930504b65be34a37ac7455cda9c9c7b6" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 41259 } } peers { permanent_uuid: "40b11f4fbad9437eb9f11d2da5dd0d8b" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 44525 } }
I20260806 06:31:31.456156 18121 leader_election.cc:288] T 3656116260e94f3ab2daf2b235e0446e P 930504b65be34a37ac7455cda9c9c7b6 [CANDIDATE]: Term 1 election: Requested vote from peers 17fc103e74544c2a98d06b27aabb9435 (127.7.43.65:39903), 40b11f4fbad9437eb9f11d2da5dd0d8b (127.7.43.66:44525)
I20260806 06:31:31.456281 17773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3656116260e94f3ab2daf2b235e0446e" candidate_uuid: "930504b65be34a37ac7455cda9c9c7b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40b11f4fbad9437eb9f11d2da5dd0d8b"
I20260806 06:31:31.456277 17442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3656116260e94f3ab2daf2b235e0446e" candidate_uuid: "930504b65be34a37ac7455cda9c9c7b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17fc103e74544c2a98d06b27aabb9435"
I20260806 06:31:31.456348 17442 raft_consensus.cc:3037] T 3656116260e94f3ab2daf2b235e0446e P 17fc103e74544c2a98d06b27aabb9435 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:31.456348 17773 raft_consensus.cc:3037] T 3656116260e94f3ab2daf2b235e0446e P 40b11f4fbad9437eb9f11d2da5dd0d8b [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:31.457033 17442 raft_consensus.cc:2454] T 3656116260e94f3ab2daf2b235e0446e P 17fc103e74544c2a98d06b27aabb9435 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 930504b65be34a37ac7455cda9c9c7b6 in term 1.
I20260806 06:31:31.457201 17773 raft_consensus.cc:2454] T 3656116260e94f3ab2daf2b235e0446e P 40b11f4fbad9437eb9f11d2da5dd0d8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 930504b65be34a37ac7455cda9c9c7b6 in term 1.
I20260806 06:31:31.457278 17878 leader_election.cc:302] T 3656116260e94f3ab2daf2b235e0446e P 930504b65be34a37ac7455cda9c9c7b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17fc103e74544c2a98d06b27aabb9435, 930504b65be34a37ac7455cda9c9c7b6; no voters:
I20260806 06:31:31.457366 18121 raft_consensus.cc:2781] T 3656116260e94f3ab2daf2b235e0446e P 930504b65be34a37ac7455cda9c9c7b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:31:31.457508 18121 raft_consensus.cc:686] T 3656116260e94f3ab2daf2b235e0446e P 930504b65be34a37ac7455cda9c9c7b6 [term 1 LEADER]: Becoming Leader. State: Replica: 930504b65be34a37ac7455cda9c9c7b6, State: Running, Role: LEADER
I20260806 06:31:31.457577 18121 consensus_queue.cc:237] T 3656116260e94f3ab2daf2b235e0446e P 930504b65be34a37ac7455cda9c9c7b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17fc103e74544c2a98d06b27aabb9435" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 39903 } } peers { permanent_uuid: "930504b65be34a37ac7455cda9c9c7b6" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 41259 } } peers { permanent_uuid: "40b11f4fbad9437eb9f11d2da5dd0d8b" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 44525 } }
I20260806 06:31:31.458261 16142 catalog_manager.cc:5495] T 3656116260e94f3ab2daf2b235e0446e P 930504b65be34a37ac7455cda9c9c7b6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 930504b65be34a37ac7455cda9c9c7b6 (127.7.43.67). New cstate: current_term: 1 leader_uuid: "930504b65be34a37ac7455cda9c9c7b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17fc103e74544c2a98d06b27aabb9435" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 39903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "930504b65be34a37ac7455cda9c9c7b6" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 41259 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40b11f4fbad9437eb9f11d2da5dd0d8b" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 44525 } health_report { overall_health: UNKNOWN } } }
I20260806 06:31:31.468694 18100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:31.466560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45134 (local address 127.7.43.67:41259)
0806 06:31:31.466613 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:31.466615 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:31.466977 (+ 362us) server_negotiation.cc:407] Connection header received
0806 06:31:31.467023 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:31.467023 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:31.467036 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:31.467089 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:31:31.467172 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.467580 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.468201 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.468363 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.468401 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:31.468504 (+ 103us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:31:31.468578 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:31:31.468627 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:31.468642 (+ 15us) server_negotiation.cc:299] Negotiation successful
0806 06:31:31.468655 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 06:31:31.470104 17773 raft_consensus.cc:1261] T 3656116260e94f3ab2daf2b235e0446e P 40b11f4fbad9437eb9f11d2da5dd0d8b [term 1 FOLLOWER]: Refusing update from remote peer 930504b65be34a37ac7455cda9c9c7b6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:31:31.470108 17442 raft_consensus.cc:1261] T 3656116260e94f3ab2daf2b235e0446e P 17fc103e74544c2a98d06b27aabb9435 [term 1 FOLLOWER]: Refusing update from remote peer 930504b65be34a37ac7455cda9c9c7b6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:31:31.470413 18121 consensus_queue.cc:1035] T 3656116260e94f3ab2daf2b235e0446e P 930504b65be34a37ac7455cda9c9c7b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17fc103e74544c2a98d06b27aabb9435" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 39903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:31:31.470534 18126 consensus_queue.cc:1035] T 3656116260e94f3ab2daf2b235e0446e P 930504b65be34a37ac7455cda9c9c7b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b11f4fbad9437eb9f11d2da5dd0d8b" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 44525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:31:31.473125 18135 mvcc.cc:205] Tried to move back new op lower bound from 7315447363459641344 to 7315447363410788352. Current Snapshot: MvccSnapshot[applied={T|T < 7315447363459641344}]
I20260806 06:31:31.474960 18137 mvcc.cc:205] Tried to move back new op lower bound from 7315447363459641344 to 7315447363410788352. Current Snapshot: MvccSnapshot[applied={T|T < 7315447363459641344 or (T in {7315447363459641344})}]
I20260806 06:31:31.475162 16141 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45684:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:31:31.475257 16141 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45684:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:31:31.476462 16141 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee7ac2b4afd849459e289fe5dba19836: Sending DeleteTablet for 3 replicas of tablet 3656116260e94f3ab2daf2b235e0446e
I20260806 06:31:31.476778 17745 tablet_service.cc:1514] Processing DeleteTablet for tablet 3656116260e94f3ab2daf2b235e0446e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:31:31 UTC) from {username='oryx', principal='oryx/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:55344
I20260806 06:31:31.476992 18149 tablet_replica.cc:330] T 3656116260e94f3ab2daf2b235e0446e P 40b11f4fbad9437eb9f11d2da5dd0d8b: stopping tablet replica
I20260806 06:31:31.476992 17955 tablet_service.cc:1514] Processing DeleteTablet for tablet 3656116260e94f3ab2daf2b235e0446e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:31:31 UTC) from {username='oryx', principal='oryx/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:45130
I20260806 06:31:31.477087 18149 raft_consensus.cc:2229] T 3656116260e94f3ab2daf2b235e0446e P 40b11f4fbad9437eb9f11d2da5dd0d8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:31:31.477181 18149 raft_consensus.cc:2258] T 3656116260e94f3ab2daf2b235e0446e P 40b11f4fbad9437eb9f11d2da5dd0d8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:31:31.477186 17411 tablet_service.cc:1514] Processing DeleteTablet for tablet 3656116260e94f3ab2daf2b235e0446e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:31:31 UTC) from {username='oryx', principal='oryx/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:40326
I20260806 06:31:31.477502 18153 tablet_replica.cc:330] T 3656116260e94f3ab2daf2b235e0446e P 17fc103e74544c2a98d06b27aabb9435: stopping tablet replica
I20260806 06:31:31.477592 18153 raft_consensus.cc:2229] T 3656116260e94f3ab2daf2b235e0446e P 17fc103e74544c2a98d06b27aabb9435 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:31:31.477638 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 16094
I20260806 06:31:31.477708 18153 raft_consensus.cc:2258] T 3656116260e94f3ab2daf2b235e0446e P 17fc103e74544c2a98d06b27aabb9435 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:31:31.477752 18149 ts_tablet_manager.cc:1905] T 3656116260e94f3ab2daf2b235e0446e P 40b11f4fbad9437eb9f11d2da5dd0d8b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:31:31.483476 18153 ts_tablet_manager.cc:1905] T 3656116260e94f3ab2daf2b235e0446e P 17fc103e74544c2a98d06b27aabb9435: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:31:31.490442 18151 tablet_replica.cc:330] T 3656116260e94f3ab2daf2b235e0446e P 930504b65be34a37ac7455cda9c9c7b6: stopping tablet replica
I20260806 06:31:31.490548 18151 raft_consensus.cc:2229] T 3656116260e94f3ab2daf2b235e0446e P 930504b65be34a37ac7455cda9c9c7b6 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:31:31.490664 18151 raft_consensus.cc:2258] T 3656116260e94f3ab2daf2b235e0446e P 930504b65be34a37ac7455cda9c9c7b6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:31:31.491190 18151 ts_tablet_manager.cc:1905] T 3656116260e94f3ab2daf2b235e0446e P 930504b65be34a37ac7455cda9c9c7b6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:31:31.492457 17149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:31.477441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40364 (local address 127.7.43.125:34313)
0806 06:31:31.477478 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:31.477480 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:31.484817 (+ 7337us) server_negotiation.cc:407] Connection header received
0806 06:31:31.484932 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:31.484933 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:31.484947 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:31.485014 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:31.485929 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.486681 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.487645 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.487937 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.488008 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:31.489495 (+ 1487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:31.489497 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:31.489497 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:31.489516 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:31.491099 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:31.491634 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:31.491635 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:31.491636 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:31.491694 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:31.491881 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:31.491882 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:31.491882 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:31.492001 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:31.492071 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:31.492264 (+ 193us) server_negotiation.cc:299] Negotiation successful
0806 06:31:31.492368 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 06:31:31.493768 18149 ts_tablet_manager.cc:1918] T 3656116260e94f3ab2daf2b235e0446e P 40b11f4fbad9437eb9f11d2da5dd0d8b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:31:31.493862 18149 log.cc:1196] T 3656116260e94f3ab2daf2b235e0446e P 40b11f4fbad9437eb9f11d2da5dd0d8b: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-1/wal/wals/3656116260e94f3ab2daf2b235e0446e
I20260806 06:31:31.494127 18149 ts_tablet_manager.cc:1939] T 3656116260e94f3ab2daf2b235e0446e P 40b11f4fbad9437eb9f11d2da5dd0d8b: Deleting consensus metadata
W20260806 06:31:31.494812 17663 connection.cc:441] server connection from 127.0.0.1:55344 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55344 (request call id 1) could send its response
I20260806 06:31:31.494858 18153 ts_tablet_manager.cc:1918] T 3656116260e94f3ab2daf2b235e0446e P 17fc103e74544c2a98d06b27aabb9435: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:31:31.494931 18153 log.cc:1196] T 3656116260e94f3ab2daf2b235e0446e P 17fc103e74544c2a98d06b27aabb9435: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-0/wal/wals/3656116260e94f3ab2daf2b235e0446e
I20260806 06:31:31.495167 18153 ts_tablet_manager.cc:1939] T 3656116260e94f3ab2daf2b235e0446e P 17fc103e74544c2a98d06b27aabb9435: Deleting consensus metadata
I20260806 06:31:31.495545 18151 ts_tablet_manager.cc:1918] T 3656116260e94f3ab2daf2b235e0446e P 930504b65be34a37ac7455cda9c9c7b6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:31:31.495617 18151 log.cc:1196] T 3656116260e94f3ab2daf2b235e0446e P 930504b65be34a37ac7455cda9c9c7b6: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-2/wal/wals/3656116260e94f3ab2daf2b235e0446e
I20260806 06:31:31.495851 18151 ts_tablet_manager.cc:1939] T 3656116260e94f3ab2daf2b235e0446e P 930504b65be34a37ac7455cda9c9c7b6: Deleting consensus metadata
W20260806 06:31:31.496160 17319 connection.cc:441] server connection from 127.0.0.1:40326 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40326 (request call id 1) could send its response
W20260806 06:31:31.496263 17884 connection.cc:441] server connection from 127.0.0.1:45130 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45130 (request call id 1) could send its response
I20260806 06:31:31.501123 17149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:31.493574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40372 (local address 127.7.43.125:34313)
0806 06:31:31.493610 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:31.493612 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:31.494014 (+ 402us) server_negotiation.cc:407] Connection header received
0806 06:31:31.494128 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:31.494129 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:31.494138 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:31.494163 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:31.495025 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.495671 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.496601 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.496899 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.496965 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:31.498180 (+ 1215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:31.498181 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:31.498181 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:31.498188 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:31.499629 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:31.500287 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:31.500288 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:31.500288 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:31.500346 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:31.500546 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:31.500547 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:31.500547 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:31.500661 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:31.500726 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:31.500954 (+ 228us) server_negotiation.cc:299] Negotiation successful
0806 06:31:31.501025 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:31:31.501163 17848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:31.477269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58752 (local address 127.7.43.124:45905)
0806 06:31:31.477305 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:31.477307 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:31.492800 (+ 15493us) server_negotiation.cc:407] Connection header received
0806 06:31:31.493121 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:31.493122 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:31.493133 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:31.493183 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:31.493859 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.494583 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.496021 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.496328 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.496364 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:31.497943 (+ 1579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:31.497944 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:31.497945 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:31.497952 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:31.499732 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:31.500324 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:31.500325 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:31.500326 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:31.500376 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:31.500550 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:31.500551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:31.500551 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:31.500672 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:31.500729 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:31.500954 (+ 225us) server_negotiation.cc:299] Negotiation successful
0806 06:31:31.501050 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:31:31.503826 18163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:31.493222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58758 (local address 127.7.43.124:45905)
0806 06:31:31.496641 (+ 3419us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:31.496643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:31.496652 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:31:31.496670 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:31.496671 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:31.496681 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:31.496753 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:31.497384 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.498185 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.499228 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:31.499598 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:31.499686 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:31.500920 (+ 1234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:31.500922 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:31.500922 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:31.500928 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:31.502902 (+ 1974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:31.503180 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:31.503181 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:31.503181 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:31.503238 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:31.503383 (+ 145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:31.503384 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:31.503385 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:31.503501 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:31.503590 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:31.503645 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 06:31:31.503722 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3393,"thread_start_us":101,"threads_started":1}
I20260806 06:31:32.474479 18168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:32.474219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:36787 (local address 127.7.43.66:33661)
0806 06:31:32.474362 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:32.474437 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:36787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":66,"threads_started":1}
W20260806 06:31:32.474774 17841 heartbeater.cc:644] Failed to heartbeat to 127.7.43.126:36787 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.43.126:36787: connect: Connection refused (error 111)
I20260806 06:31:32.475041 18169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:32.474820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:36787 (local address 127.7.43.67:35853)
0806 06:31:32.474925 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:32.474998 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:36787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":54,"threads_started":1}
W20260806 06:31:32.475368 18061 heartbeater.cc:644] Failed to heartbeat to 127.7.43.126:36787 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.43.126:36787: connect: Connection refused (error 111)
I20260806 06:31:32.475643 18170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:32.475426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:36787 (local address 127.7.43.65:47143)
0806 06:31:32.475545 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:32.475587 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:36787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":54,"threads_started":1}
I20260806 06:31:32.977041 18172 raft_consensus.cc:482] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ee7ac2b4afd849459e289fe5dba19836)
I20260806 06:31:32.977174 18172 raft_consensus.cc:504] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } } peers { permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 34313 } attrs { promote: false } } peers { permanent_uuid: "cbb47ccf5ac7455dbb8898888f83dbf3" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 45905 } attrs { promote: false } }
I20260806 06:31:32.977393 18172 leader_election.cc:288] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ee7ac2b4afd849459e289fe5dba19836 (127.7.43.126:36787), bb5fccea882c4272838ffd2bb3d4a023 (127.7.43.125:34313)
I20260806 06:31:32.977504 18173 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ee7ac2b4afd849459e289fe5dba19836)
I20260806 06:31:32.977649 18173 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } } peers { permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 34313 } attrs { promote: false } } peers { permanent_uuid: "cbb47ccf5ac7455dbb8898888f83dbf3" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 45905 } attrs { promote: false } }
I20260806 06:31:32.977746 18174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:32.977508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:36787 (local address 127.0.0.1:46128)
0806 06:31:32.977604 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:32.977691 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:36787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77,"thread_start_us":49,"threads_started":1}
W20260806 06:31:32.977908 17549 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.43.126:36787: connect: Connection refused (error 111)
I20260806 06:31:32.978300 18174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:32.978208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:36787 (local address 127.0.0.1:46152)
0806 06:31:32.978248 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:32.978271 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:36787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260806 06:31:32.978363 18184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:32.978117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:36787 (local address 127.0.0.1:46140)
0806 06:31:32.978235 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:32.978294 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:36787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":45,"threads_started":1}
W20260806 06:31:32.978473 17049 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.43.126:36787: connect: Connection refused (error 111)
I20260806 06:31:32.978493 18173 leader_election.cc:288] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ee7ac2b4afd849459e289fe5dba19836 (127.7.43.126:36787), cbb47ccf5ac7455dbb8898888f83dbf3 (127.7.43.124:45905)
W20260806 06:31:32.978550 17549 leader_election.cc:334] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ee7ac2b4afd849459e289fe5dba19836 (127.7.43.126:36787): Network error: Client connection negotiation failed: client connection to 127.7.43.126:36787: connect: Connection refused (error 111)
I20260806 06:31:32.978958 18184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:32.978860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:36787 (local address 127.0.0.1:46154)
0806 06:31:32.978887 (+ 27us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:32.978914 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:36787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260806 06:31:32.979041 17049 leader_election.cc:334] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ee7ac2b4afd849459e289fe5dba19836 (127.7.43.126:36787): Network error: Client connection negotiation failed: client connection to 127.7.43.126:36787: connect: Connection refused (error 111)
I20260806 06:31:32.981024 18181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:32.977731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40384 (local address 127.7.43.125:34313)
0806 06:31:32.977917 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:32.977919 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:32.977928 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:31:32.977955 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:32.977956 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:32.977968 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:32.978036 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:31:32.978201 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:32.979020 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:32.980173 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:32.980719 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:32.980786 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:32.980798 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:32.980917 (+ 119us) server_negotiation.cc:299] Negotiation successful
0806 06:31:32.980959 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":167,"thread_start_us":56,"threads_started":1}
I20260806 06:31:32.981019 18178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:32.977671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.125:34313 (local address 127.0.0.1:40384)
0806 06:31:32.977819 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:32.977829 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:32.977888 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:32.978056 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:32.978057 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:32.978060 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:31:32.978126 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:32.978128 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:32.979118 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:32.979119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:32.980073 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:32.980076 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:32.980794 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:32.980795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:32.980853 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:32.980858 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:31:32.980900 (+ 42us) client_negotiation.cc:240] Negotiation successful
0806 06:31:32.980939 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":67,"threads_started":1}
I20260806 06:31:32.981246 17084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cbb47ccf5ac7455dbb8898888f83dbf3" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "bb5fccea882c4272838ffd2bb3d4a023" is_pre_election: true
I20260806 06:31:32.981340 17084 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cbb47ccf5ac7455dbb8898888f83dbf3 in term 1.
I20260806 06:31:32.981484 17549 leader_election.cc:302] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bb5fccea882c4272838ffd2bb3d4a023, cbb47ccf5ac7455dbb8898888f83dbf3; no voters: ee7ac2b4afd849459e289fe5dba19836
I20260806 06:31:32.981603 18172 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 06:31:32.981653 18172 raft_consensus.cc:482] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ee7ac2b4afd849459e289fe5dba19836)
I20260806 06:31:32.981678 18172 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3 [term 1 FOLLOWER]: Advancing to term 2
I20260806 06:31:32.981760 18187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:32.978605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58772 (local address 127.7.43.124:45905)
0806 06:31:32.978765 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:32.978767 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:32.978776 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:31:32.978797 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:32.978798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:32.978808 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:32.978868 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:31:32.979105 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:32.979654 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:32.980961 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:32.981460 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:32.981511 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:32.981520 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:32.981655 (+ 135us) server_negotiation.cc:299] Negotiation successful
0806 06:31:32.981697 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":138,"thread_start_us":74,"threads_started":1}
I20260806 06:31:32.981770 18185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:32.978297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.124:45905 (local address 127.0.0.1:58772)
0806 06:31:32.978645 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:32.978655 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:32.978730 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:32.978914 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:32.978915 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:32.978917 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:31:32.979016 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:32.979019 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:32.979733 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:32.979742 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:32.980887 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:32.980889 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:32.981526 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:32.981526 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:32.981599 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:32.981603 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:31:32.981650 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 06:31:32.981690 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"mutex_wait_us":2,"thread_start_us":172,"threads_started":1}
I20260806 06:31:32.982028 17583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bb5fccea882c4272838ffd2bb3d4a023" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "cbb47ccf5ac7455dbb8898888f83dbf3" is_pre_election: true
I20260806 06:31:32.982482 18172 raft_consensus.cc:504] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } } peers { permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 34313 } attrs { promote: false } } peers { permanent_uuid: "cbb47ccf5ac7455dbb8898888f83dbf3" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 45905 } attrs { promote: false } }
I20260806 06:31:32.982615 17583 raft_consensus.cc:2379] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bb5fccea882c4272838ffd2bb3d4a023 in current term 2: Already voted for candidate cbb47ccf5ac7455dbb8898888f83dbf3 in this term.
I20260806 06:31:32.982631 18172 leader_election.cc:288] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3 [CANDIDATE]: Term 2 election: Requested vote from peers ee7ac2b4afd849459e289fe5dba19836 (127.7.43.126:36787), bb5fccea882c4272838ffd2bb3d4a023 (127.7.43.125:34313)
I20260806 06:31:32.982800 18178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:32.982707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:36787 (local address 127.0.0.1:46168)
0806 06:31:32.982745 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:32.982778 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:36787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260806 06:31:32.982836 17084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cbb47ccf5ac7455dbb8898888f83dbf3" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "bb5fccea882c4272838ffd2bb3d4a023"
I20260806 06:31:32.982890 17047 leader_election.cc:302] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb5fccea882c4272838ffd2bb3d4a023; no voters: cbb47ccf5ac7455dbb8898888f83dbf3, ee7ac2b4afd849459e289fe5dba19836
I20260806 06:31:32.982905 17084 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023 [term 1 FOLLOWER]: Advancing to term 2
I20260806 06:31:32.983067 18178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:32.982994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:36787 (local address 127.0.0.1:46184)
0806 06:31:32.983028 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:32.983048 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:36787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260806 06:31:32.983153 17549 leader_election.cc:334] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer ee7ac2b4afd849459e289fe5dba19836 (127.7.43.126:36787): Network error: Client connection negotiation failed: client connection to 127.7.43.126:36787: connect: Connection refused (error 111)
I20260806 06:31:32.983604 17084 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cbb47ccf5ac7455dbb8898888f83dbf3 in term 2.
I20260806 06:31:32.983673 18173 raft_consensus.cc:2726] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 06:31:32.983778 17549 leader_election.cc:302] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bb5fccea882c4272838ffd2bb3d4a023, cbb47ccf5ac7455dbb8898888f83dbf3; no voters: ee7ac2b4afd849459e289fe5dba19836
I20260806 06:31:32.983851 18172 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 06:31:32.983981 18172 raft_consensus.cc:686] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3 [term 2 LEADER]: Becoming Leader. State: Replica: cbb47ccf5ac7455dbb8898888f83dbf3, State: Running, Role: LEADER
I20260806 06:31:32.984037 18172 consensus_queue.cc:237] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } } peers { permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 34313 } attrs { promote: false } } peers { permanent_uuid: "cbb47ccf5ac7455dbb8898888f83dbf3" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 45905 } attrs { promote: false } }
I20260806 06:31:32.984318 18188 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cbb47ccf5ac7455dbb8898888f83dbf3. Latest consensus state: current_term: 2 leader_uuid: "cbb47ccf5ac7455dbb8898888f83dbf3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } } peers { permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 34313 } attrs { promote: false } } peers { permanent_uuid: "cbb47ccf5ac7455dbb8898888f83dbf3" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 45905 } attrs { promote: false } } }
I20260806 06:31:32.984390 18188 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3 [sys.catalog]: This master's current role is: LEADER
I20260806 06:31:32.984527 18192 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:31:32.984948 18192 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:31:32.985030 18192 catalog_manager.cc:1259] Loaded cluster ID: f88ac3451d944ee6994511cbff6c6a75
I20260806 06:31:32.985060 18192 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:31:32.985200 18192 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:31:32.985278 18192 catalog_manager.cc:5879] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3: Loaded TSK: 0
I20260806 06:31:32.985409 18192 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:31:33.025563 17564 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58758:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:31:33.027335 17564 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:31:33.028272 18178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:33.028176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:36787 (local address 127.0.0.1:46214)
0806 06:31:33.028217 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:33.028249 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:36787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260806 06:31:33.028286 17084 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023 [term 2 FOLLOWER]: Refusing update from remote peer cbb47ccf5ac7455dbb8898888f83dbf3: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260806 06:31:33.028512 18188 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 34313 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260806 06:31:33.028627 18178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:33.028534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:36787 (local address 127.0.0.1:46228)
0806 06:31:33.028576 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:33.028602 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:36787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260806 06:31:33.028710 17549 consensus_peers.cc:487] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3 -> Peer ee7ac2b4afd849459e289fe5dba19836 (127.7.43.126:36787): Couldn't send request to peer ee7ac2b4afd849459e289fe5dba19836. Status: Network error: Client connection negotiation failed: client connection to 127.7.43.126:36787: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 06:31:33.029071 18173 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cbb47ccf5ac7455dbb8898888f83dbf3. Latest consensus state: current_term: 2 leader_uuid: "cbb47ccf5ac7455dbb8898888f83dbf3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } } peers { permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 34313 } attrs { promote: false } } peers { permanent_uuid: "cbb47ccf5ac7455dbb8898888f83dbf3" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 45905 } attrs { promote: false } } }
I20260806 06:31:33.029156 18173 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:31:33.029440 18196 mvcc.cc:205] Tried to move back new op lower bound from 7315447369841930240 to 7315447369662976000. Current Snapshot: MvccSnapshot[applied={T|T < 7315447369841930240}]
I20260806 06:31:33.029534 18172 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "cbb47ccf5ac7455dbb8898888f83dbf3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } } peers { permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 34313 } attrs { promote: false } } peers { permanent_uuid: "cbb47ccf5ac7455dbb8898888f83dbf3" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 45905 } attrs { promote: false } } }
I20260806 06:31:33.029618 18172 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3 [sys.catalog]: This master's current role is: LEADER
I20260806 06:31:33.029851 18173 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "cbb47ccf5ac7455dbb8898888f83dbf3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ee7ac2b4afd849459e289fe5dba19836" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36787 } } peers { permanent_uuid: "bb5fccea882c4272838ffd2bb3d4a023" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 34313 } attrs { promote: false } } peers { permanent_uuid: "cbb47ccf5ac7455dbb8898888f83dbf3" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 45905 } attrs { promote: false } } }
I20260806 06:31:33.029935 18173 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb5fccea882c4272838ffd2bb3d4a023 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:31:33.034099 18210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:33.031361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45148 (local address 127.7.43.67:41259)
0806 06:31:33.031472 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:33.031474 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:33.031484 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:31:33.031507 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:33.031508 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:33.031518 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:33.031577 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:31:33.031727 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:33.032425 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:33.033273 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:33.033767 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:33.033835 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:33.033847 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:33.033995 (+ 148us) server_negotiation.cc:299] Negotiation successful
0806 06:31:33.034030 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":57,"threads_started":1}
I20260806 06:31:33.034107 18205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:33.031178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.67:41259 (local address 127.0.0.1:45148)
0806 06:31:33.031289 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:33.031298 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:33.031449 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:33.031578 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:33.031578 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:33.031580 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:31:33.031637 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:33.031638 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:33.032482 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:33.032483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:33.033194 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:33.033197 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:33.033843 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:33.033843 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:33.033926 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:33.033931 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:31:33.033984 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 06:31:33.034021 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":78,"threads_started":1}
I20260806 06:31:33.034323 17955 tablet_service.cc:1467] Processing CreateTablet for tablet 2d65d318d8ff486597dac22338e0bb37 (DEFAULT_TABLE table=test-table [id=a02e3504c3c24574a6e878f0197cbb25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:31:33.034430 18174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:33.031083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.66:44525 (local address 127.0.0.1:55356)
0806 06:31:33.031141 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:33.031151 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:33.031203 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:33.031502 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:33.031503 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:33.031505 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:31:33.031566 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:33.031567 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:33.032289 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:33.032290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:33.033409 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:33.033411 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:33.034124 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:33.034125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:33.034209 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:33.034213 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:31:33.034253 (+ 40us) client_negotiation.cc:240] Negotiation successful
0806 06:31:33.034292 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260806 06:31:33.034471 17955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d65d318d8ff486597dac22338e0bb37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:33.034546 18204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:33.031151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40330 (local address 127.7.43.65:39903)
0806 06:31:33.031273 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:33.031275 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:33.031282 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:31:33.031307 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:33.031308 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:33.031318 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:33.031379 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:31:33.031519 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:33.032349 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:33.033521 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:33.034093 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:33.034404 (+ 311us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:33.034421 (+ 17us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:33.034440 (+ 19us) server_negotiation.cc:299] Negotiation successful
0806 06:31:33.034485 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":59,"threads_started":1}
I20260806 06:31:33.034605 18178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:33.031075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:39903 (local address 127.0.0.1:40330)
0806 06:31:33.031113 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:33.031122 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:33.031155 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:33.031383 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:33.031385 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:33.031386 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:31:33.031430 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:33.031431 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:33.032420 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:33.032420 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:33.033437 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:33.033438 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:33.034181 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:33.034181 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:33.034239 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:33.034245 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 06:31:33.034273 (+ 28us) client_negotiation.cc:240] Negotiation successful
0806 06:31:33.034306 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260806 06:31:33.034833 18209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:33.031280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55356 (local address 127.7.43.66:44525)
0806 06:31:33.031373 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:33.031375 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:33.031383 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:31:33.031405 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:33.031406 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:33.031415 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:33.031625 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:31:33.031661 (+ 36us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:33.032217 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:33.033497 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:33.034030 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:33.034566 (+ 536us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:33.034582 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:33.034720 (+ 138us) server_negotiation.cc:299] Negotiation successful
0806 06:31:33.034766 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77,"thread_start_us":46,"threads_started":1}
I20260806 06:31:33.034868 17411 tablet_service.cc:1467] Processing CreateTablet for tablet 2d65d318d8ff486597dac22338e0bb37 (DEFAULT_TABLE table=test-table [id=a02e3504c3c24574a6e878f0197cbb25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:31:33.034978 17411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d65d318d8ff486597dac22338e0bb37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:33.035089 17745 tablet_service.cc:1467] Processing CreateTablet for tablet 2d65d318d8ff486597dac22338e0bb37 (DEFAULT_TABLE table=test-table [id=a02e3504c3c24574a6e878f0197cbb25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:31:33.035213 17745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d65d318d8ff486597dac22338e0bb37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:33.035876 18212 tablet_bootstrap.cc:492] T 2d65d318d8ff486597dac22338e0bb37 P 930504b65be34a37ac7455cda9c9c7b6: Bootstrap starting.
I20260806 06:31:33.036300 18212 tablet_bootstrap.cc:654] T 2d65d318d8ff486597dac22338e0bb37 P 930504b65be34a37ac7455cda9c9c7b6: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:33.036466 18213 tablet_bootstrap.cc:492] T 2d65d318d8ff486597dac22338e0bb37 P 17fc103e74544c2a98d06b27aabb9435: Bootstrap starting.
I20260806 06:31:33.036623 18214 tablet_bootstrap.cc:492] T 2d65d318d8ff486597dac22338e0bb37 P 40b11f4fbad9437eb9f11d2da5dd0d8b: Bootstrap starting.
I20260806 06:31:33.036895 18213 tablet_bootstrap.cc:654] T 2d65d318d8ff486597dac22338e0bb37 P 17fc103e74544c2a98d06b27aabb9435: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:33.036908 18212 tablet_bootstrap.cc:492] T 2d65d318d8ff486597dac22338e0bb37 P 930504b65be34a37ac7455cda9c9c7b6: No bootstrap required, opened a new log
I20260806 06:31:33.036948 18212 ts_tablet_manager.cc:1397] T 2d65d318d8ff486597dac22338e0bb37 P 930504b65be34a37ac7455cda9c9c7b6: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:31:33.036986 18214 tablet_bootstrap.cc:654] T 2d65d318d8ff486597dac22338e0bb37 P 40b11f4fbad9437eb9f11d2da5dd0d8b: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:33.037047 18212 raft_consensus.cc:348] T 2d65d318d8ff486597dac22338e0bb37 P 930504b65be34a37ac7455cda9c9c7b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17fc103e74544c2a98d06b27aabb9435" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 39903 } } peers { permanent_uuid: "40b11f4fbad9437eb9f11d2da5dd0d8b" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 44525 } } peers { permanent_uuid: "930504b65be34a37ac7455cda9c9c7b6" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 41259 } }
I20260806 06:31:33.037117 18212 raft_consensus.cc:374] T 2d65d318d8ff486597dac22338e0bb37 P 930504b65be34a37ac7455cda9c9c7b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:33.037138 18212 raft_consensus.cc:729] T 2d65d318d8ff486597dac22338e0bb37 P 930504b65be34a37ac7455cda9c9c7b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 930504b65be34a37ac7455cda9c9c7b6, State: Initialized, Role: FOLLOWER
I20260806 06:31:33.037185 18212 consensus_queue.cc:260] T 2d65d318d8ff486597dac22338e0bb37 P 930504b65be34a37ac7455cda9c9c7b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17fc103e74544c2a98d06b27aabb9435" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 39903 } } peers { permanent_uuid: "40b11f4fbad9437eb9f11d2da5dd0d8b" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 44525 } } peers { permanent_uuid: "930504b65be34a37ac7455cda9c9c7b6" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 41259 } }
I20260806 06:31:33.037345 18212 ts_tablet_manager.cc:1428] T 2d65d318d8ff486597dac22338e0bb37 P 930504b65be34a37ac7455cda9c9c7b6: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 06:31:33.037495 18213 tablet_bootstrap.cc:492] T 2d65d318d8ff486597dac22338e0bb37 P 17fc103e74544c2a98d06b27aabb9435: No bootstrap required, opened a new log
I20260806 06:31:33.037494 18214 tablet_bootstrap.cc:492] T 2d65d318d8ff486597dac22338e0bb37 P 40b11f4fbad9437eb9f11d2da5dd0d8b: No bootstrap required, opened a new log
I20260806 06:31:33.037544 18214 ts_tablet_manager.cc:1397] T 2d65d318d8ff486597dac22338e0bb37 P 40b11f4fbad9437eb9f11d2da5dd0d8b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:31:33.037544 18213 ts_tablet_manager.cc:1397] T 2d65d318d8ff486597dac22338e0bb37 P 17fc103e74544c2a98d06b27aabb9435: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:31:33.037668 18213 raft_consensus.cc:348] T 2d65d318d8ff486597dac22338e0bb37 P 17fc103e74544c2a98d06b27aabb9435 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17fc103e74544c2a98d06b27aabb9435" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 39903 } } peers { permanent_uuid: "40b11f4fbad9437eb9f11d2da5dd0d8b" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 44525 } } peers { permanent_uuid: "930504b65be34a37ac7455cda9c9c7b6" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 41259 } }
I20260806 06:31:33.037729 18213 raft_consensus.cc:374] T 2d65d318d8ff486597dac22338e0bb37 P 17fc103e74544c2a98d06b27aabb9435 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:33.037671 18214 raft_consensus.cc:348] T 2d65d318d8ff486597dac22338e0bb37 P 40b11f4fbad9437eb9f11d2da5dd0d8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17fc103e74544c2a98d06b27aabb9435" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 39903 } } peers { permanent_uuid: "40b11f4fbad9437eb9f11d2da5dd0d8b" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 44525 } } peers { permanent_uuid: "930504b65be34a37ac7455cda9c9c7b6" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 41259 } }
I20260806 06:31:33.037746 18213 raft_consensus.cc:729] T 2d65d318d8ff486597dac22338e0bb37 P 17fc103e74544c2a98d06b27aabb9435 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17fc103e74544c2a98d06b27aabb9435, State: Initialized, Role: FOLLOWER
I20260806 06:31:33.037751 18214 raft_consensus.cc:374] T 2d65d318d8ff486597dac22338e0bb37 P 40b11f4fbad9437eb9f11d2da5dd0d8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:33.037771 18214 raft_consensus.cc:729] T 2d65d318d8ff486597dac22338e0bb37 P 40b11f4fbad9437eb9f11d2da5dd0d8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40b11f4fbad9437eb9f11d2da5dd0d8b, State: Initialized, Role: FOLLOWER
I20260806 06:31:33.037786 18213 consensus_queue.cc:260] T 2d65d318d8ff486597dac22338e0bb37 P 17fc103e74544c2a98d06b27aabb9435 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17fc103e74544c2a98d06b27aabb9435" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 39903 } } peers { permanent_uuid: "40b11f4fbad9437eb9f11d2da5dd0d8b" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 44525 } } peers { permanent_uuid: "930504b65be34a37ac7455cda9c9c7b6" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 41259 } }
I20260806 06:31:33.037801 18221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:33.037601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:36787 (local address 127.7.43.67:37563)
0806 06:31:33.037731 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:33.037770 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:36787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":76,"threads_started":1}
I20260806 06:31:33.037807 18214 consensus_queue.cc:260] T 2d65d318d8ff486597dac22338e0bb37 P 40b11f4fbad9437eb9f11d2da5dd0d8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17fc103e74544c2a98d06b27aabb9435" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 39903 } } peers { permanent_uuid: "40b11f4fbad9437eb9f11d2da5dd0d8b" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 44525 } } peers { permanent_uuid: "930504b65be34a37ac7455cda9c9c7b6" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 41259 } }
I20260806 06:31:33.037931 18214 ts_tablet_manager.cc:1428] T 2d65d318d8ff486597dac22338e0bb37 P 40b11f4fbad9437eb9f11d2da5dd0d8b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 06:31:33.037935 18213 ts_tablet_manager.cc:1428] T 2d65d318d8ff486597dac22338e0bb37 P 17fc103e74544c2a98d06b27aabb9435: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 06:31:33.038475 18060 heartbeater.cc:503] Master 127.7.43.124:45905 was elected leader, sending a full tablet report...
I20260806 06:31:33.038944 18221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:33.038857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:36787 (local address 127.7.43.67:59403)
0806 06:31:33.038888 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:33.038919 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:36787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260806 06:31:33.039242 18227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:33.038321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:36787 (local address 127.7.43.66:48141)
0806 06:31:33.039170 (+ 849us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:33.039202 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:36787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":828,"thread_start_us":84,"threads_started":1}
I20260806 06:31:33.039398 17840 heartbeater.cc:503] Master 127.7.43.124:45905 was elected leader, sending a full tablet report...
I20260806 06:31:33.039534 18225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:33.038148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:36787 (local address 127.7.43.65:47735)
0806 06:31:33.039443 (+ 1295us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:33.039496 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:36787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1274,"thread_start_us":71,"threads_started":1}
I20260806 06:31:33.039820 18227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:33.039498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:36787 (local address 127.7.43.66:41597)
0806 06:31:33.039769 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:33.039797 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:36787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":258}
I20260806 06:31:33.053234 18224 raft_consensus.cc:482] T 2d65d318d8ff486597dac22338e0bb37 P 40b11f4fbad9437eb9f11d2da5dd0d8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:31:33.053289 18224 raft_consensus.cc:504] T 2d65d318d8ff486597dac22338e0bb37 P 40b11f4fbad9437eb9f11d2da5dd0d8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17fc103e74544c2a98d06b27aabb9435" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 39903 } } peers { permanent_uuid: "40b11f4fbad9437eb9f11d2da5dd0d8b" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 44525 } } peers { permanent_uuid: "930504b65be34a37ac7455cda9c9c7b6" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 41259 } }
I20260806 06:31:33.053545 18224 leader_election.cc:288] T 2d65d318d8ff486597dac22338e0bb37 P 40b11f4fbad9437eb9f11d2da5dd0d8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17fc103e74544c2a98d06b27aabb9435 (127.7.43.65:39903), 930504b65be34a37ac7455cda9c9c7b6 (127.7.43.67:41259)
I20260806 06:31:33.056007 18227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:33.053646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:39903 (local address 127.7.43.66:38349)
0806 06:31:33.053788 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:33.053796 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:33.053849 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:33.053957 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:33.053957 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:33.053959 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:31:33.054022 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:33.054023 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:33.054580 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:33.054581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:33.055321 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:33.055323 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:33.055790 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:33.055791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:33.055864 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:33.055867 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:31:33.055908 (+ 41us) client_negotiation.cc:240] Negotiation successful
0806 06:31:33.055945 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22,"mutex_wait_us":2}
I20260806 06:31:33.056003 18204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:33.053693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:38349 (local address 127.7.43.65:39903)
0806 06:31:33.053729 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:33.053731 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:33.053848 (+ 117us) server_negotiation.cc:407] Connection header received
0806 06:31:33.053904 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:33.053904 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:33.053913 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:33.053950 (+ 37us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:31:33.054072 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:33.054521 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:33.055396 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:33.055720 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:33.055781 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:33.055791 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:33.055919 (+ 128us) server_negotiation.cc:299] Negotiation successful
0806 06:31:33.055948 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260806 06:31:33.056104 18210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:33.053721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:56893 (local address 127.7.43.67:41259)
0806 06:31:33.053755 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:33.053757 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:33.053863 (+ 106us) server_negotiation.cc:407] Connection header received
0806 06:31:33.053919 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:33.053920 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:33.053930 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:33.053975 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:31:33.054111 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:33.054590 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:33.055470 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:33.055807 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:33.055863 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:33.055873 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:33.056018 (+ 145us) server_negotiation.cc:299] Negotiation successful
0806 06:31:33.056050 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 06:31:33.056118 18228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:33.053646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.67:41259 (local address 127.7.43.66:56893)
0806 06:31:33.053803 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:33.053811 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:33.053864 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:33.053983 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:33.053984 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:33.053986 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:31:33.054059 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:33.054060 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:33.054673 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:33.054674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:33.055390 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:33.055392 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:33.055887 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:33.055888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:33.055960 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:33.055964 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:31:33.056007 (+ 43us) client_negotiation.cc:240] Negotiation successful
0806 06:31:33.056044 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":32,"thread_start_us":82,"threads_started":1}
I20260806 06:31:33.056187 17442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d65d318d8ff486597dac22338e0bb37" candidate_uuid: "40b11f4fbad9437eb9f11d2da5dd0d8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17fc103e74544c2a98d06b27aabb9435" is_pre_election: true
I20260806 06:31:33.056255 17442 raft_consensus.cc:2454] T 2d65d318d8ff486597dac22338e0bb37 P 17fc103e74544c2a98d06b27aabb9435 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40b11f4fbad9437eb9f11d2da5dd0d8b in term 0.
I20260806 06:31:33.056306 17987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d65d318d8ff486597dac22338e0bb37" candidate_uuid: "40b11f4fbad9437eb9f11d2da5dd0d8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "930504b65be34a37ac7455cda9c9c7b6" is_pre_election: true
I20260806 06:31:33.056412 17987 raft_consensus.cc:2454] T 2d65d318d8ff486597dac22338e0bb37 P 930504b65be34a37ac7455cda9c9c7b6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40b11f4fbad9437eb9f11d2da5dd0d8b in term 0.
I20260806 06:31:33.056427 17660 leader_election.cc:302] T 2d65d318d8ff486597dac22338e0bb37 P 40b11f4fbad9437eb9f11d2da5dd0d8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17fc103e74544c2a98d06b27aabb9435, 40b11f4fbad9437eb9f11d2da5dd0d8b; no voters:
I20260806 06:31:33.056550 18224 raft_consensus.cc:2781] T 2d65d318d8ff486597dac22338e0bb37 P 40b11f4fbad9437eb9f11d2da5dd0d8b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:31:33.056583 18224 raft_consensus.cc:482] T 2d65d318d8ff486597dac22338e0bb37 P 40b11f4fbad9437eb9f11d2da5dd0d8b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:31:33.056602 18224 raft_consensus.cc:3037] T 2d65d318d8ff486597dac22338e0bb37 P 40b11f4fbad9437eb9f11d2da5dd0d8b [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:33.057133 18224 raft_consensus.cc:504] T 2d65d318d8ff486597dac22338e0bb37 P 40b11f4fbad9437eb9f11d2da5dd0d8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17fc103e74544c2a98d06b27aabb9435" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 39903 } } peers { permanent_uuid: "40b11f4fbad9437eb9f11d2da5dd0d8b" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 44525 } } peers { permanent_uuid: "930504b65be34a37ac7455cda9c9c7b6" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 41259 } }
I20260806 06:31:33.057245 18224 leader_election.cc:288] T 2d65d318d8ff486597dac22338e0bb37 P 40b11f4fbad9437eb9f11d2da5dd0d8b [CANDIDATE]: Term 1 election: Requested vote from peers 17fc103e74544c2a98d06b27aabb9435 (127.7.43.65:39903), 930504b65be34a37ac7455cda9c9c7b6 (127.7.43.67:41259)
I20260806 06:31:33.057382 17987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d65d318d8ff486597dac22338e0bb37" candidate_uuid: "40b11f4fbad9437eb9f11d2da5dd0d8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "930504b65be34a37ac7455cda9c9c7b6"
I20260806 06:31:33.057366 17442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d65d318d8ff486597dac22338e0bb37" candidate_uuid: "40b11f4fbad9437eb9f11d2da5dd0d8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17fc103e74544c2a98d06b27aabb9435"
I20260806 06:31:33.057442 17987 raft_consensus.cc:3037] T 2d65d318d8ff486597dac22338e0bb37 P 930504b65be34a37ac7455cda9c9c7b6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:33.057456 17442 raft_consensus.cc:3037] T 2d65d318d8ff486597dac22338e0bb37 P 17fc103e74544c2a98d06b27aabb9435 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:33.057979 17987 raft_consensus.cc:2454] T 2d65d318d8ff486597dac22338e0bb37 P 930504b65be34a37ac7455cda9c9c7b6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40b11f4fbad9437eb9f11d2da5dd0d8b in term 1.
I20260806 06:31:33.058002 17442 raft_consensus.cc:2454] T 2d65d318d8ff486597dac22338e0bb37 P 17fc103e74544c2a98d06b27aabb9435 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40b11f4fbad9437eb9f11d2da5dd0d8b in term 1.
I20260806 06:31:33.058117 17661 leader_election.cc:302] T 2d65d318d8ff486597dac22338e0bb37 P 40b11f4fbad9437eb9f11d2da5dd0d8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40b11f4fbad9437eb9f11d2da5dd0d8b, 930504b65be34a37ac7455cda9c9c7b6; no voters:
I20260806 06:31:33.058199 18224 raft_consensus.cc:2781] T 2d65d318d8ff486597dac22338e0bb37 P 40b11f4fbad9437eb9f11d2da5dd0d8b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:31:33.058323 18224 raft_consensus.cc:686] T 2d65d318d8ff486597dac22338e0bb37 P 40b11f4fbad9437eb9f11d2da5dd0d8b [term 1 LEADER]: Becoming Leader. State: Replica: 40b11f4fbad9437eb9f11d2da5dd0d8b, State: Running, Role: LEADER
I20260806 06:31:33.058434 18224 consensus_queue.cc:237] T 2d65d318d8ff486597dac22338e0bb37 P 40b11f4fbad9437eb9f11d2da5dd0d8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17fc103e74544c2a98d06b27aabb9435" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 39903 } } peers { permanent_uuid: "40b11f4fbad9437eb9f11d2da5dd0d8b" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 44525 } } peers { permanent_uuid: "930504b65be34a37ac7455cda9c9c7b6" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 41259 } }
I20260806 06:31:33.058593 18228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:33.058507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:36787 (local address 127.7.43.66:38483)
0806 06:31:33.058545 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:33.058570 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:36787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260806 06:31:33.059000 17564 catalog_manager.cc:5495] T 2d65d318d8ff486597dac22338e0bb37 P 40b11f4fbad9437eb9f11d2da5dd0d8b reported cstate change: term changed from 0 to 1, leader changed from <none> to 40b11f4fbad9437eb9f11d2da5dd0d8b (127.7.43.66). New cstate: current_term: 1 leader_uuid: "40b11f4fbad9437eb9f11d2da5dd0d8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17fc103e74544c2a98d06b27aabb9435" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 39903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40b11f4fbad9437eb9f11d2da5dd0d8b" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 44525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "930504b65be34a37ac7455cda9c9c7b6" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 41259 } health_report { overall_health: UNKNOWN } } }
I20260806 06:31:33.069455 18209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:33.067410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55358 (local address 127.7.43.66:44525)
0806 06:31:33.067450 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:33.067452 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:33.067462 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:31:33.067530 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:33.067531 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:33.067547 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:33.067603 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:31:33.067722 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:33.068131 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:33.068823 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:33.068998 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:33.069047 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:33.069185 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:31:33.069274 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:31:33.069331 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:33.069383 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 06:31:33.069399 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:31:33.070245 17442 raft_consensus.cc:1261] T 2d65d318d8ff486597dac22338e0bb37 P 17fc103e74544c2a98d06b27aabb9435 [term 1 FOLLOWER]: Refusing update from remote peer 40b11f4fbad9437eb9f11d2da5dd0d8b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:31:33.070291 17987 raft_consensus.cc:1261] T 2d65d318d8ff486597dac22338e0bb37 P 930504b65be34a37ac7455cda9c9c7b6 [term 1 FOLLOWER]: Refusing update from remote peer 40b11f4fbad9437eb9f11d2da5dd0d8b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:31:33.070499 18224 consensus_queue.cc:1035] T 2d65d318d8ff486597dac22338e0bb37 P 40b11f4fbad9437eb9f11d2da5dd0d8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "17fc103e74544c2a98d06b27aabb9435" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 39903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:31:33.070619 18229 consensus_queue.cc:1035] T 2d65d318d8ff486597dac22338e0bb37 P 40b11f4fbad9437eb9f11d2da5dd0d8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "930504b65be34a37ac7455cda9c9c7b6" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 41259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:31:33.070976 17508 heartbeater.cc:503] Master 127.7.43.124:45905 was elected leader, sending a full tablet report...
I20260806 06:31:33.071158 18225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:33.071071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:36787 (local address 127.7.43.65:45745)
0806 06:31:33.071104 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:33.071135 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:36787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260806 06:31:33.071633 18221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:33.071537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:36787 (local address 127.7.43.67:53259)
0806 06:31:33.071579 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:33.071607 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:36787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260806 06:31:33.072155 18228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:33.072073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:36787 (local address 127.7.43.66:48899)
0806 06:31:33.072105 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:33.072132 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:36787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260806 06:31:33.073228 18228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:33.073138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:36787 (local address 127.7.43.66:53723)
0806 06:31:33.073170 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:33.073204 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:36787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260806 06:31:33.073248 18221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:33.073126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:36787 (local address 127.7.43.67:37175)
0806 06:31:33.073189 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:33.073213 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:36787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260806 06:31:33.073287 18225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:33.073138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:36787 (local address 127.7.43.65:47285)
0806 06:31:33.073241 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:33.073264 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:36787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260806 06:31:33.073832 17564 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58758:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:31:33.073905 17564 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58758:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:31:33.074018 18238 mvcc.cc:205] Tried to move back new op lower bound from 7315447370014146560 to 7315447369967747072. Current Snapshot: MvccSnapshot[applied={T|T < 7315447370014146560 or (T in {7315447370014146560})}]
I20260806 06:31:33.075021 17564 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cbb47ccf5ac7455dbb8898888f83dbf3: Sending DeleteTablet for 3 replicas of tablet 2d65d318d8ff486597dac22338e0bb37
I20260806 06:31:33.075304 17411 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d65d318d8ff486597dac22338e0bb37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:31:33 UTC) from {username='oryx', principal='oryx/127.7.43.124@KRBTEST.COM'} at 127.0.0.1:40330
I20260806 06:31:33.075488 18245 tablet_replica.cc:330] T 2d65d318d8ff486597dac22338e0bb37 P 17fc103e74544c2a98d06b27aabb9435: stopping tablet replica
I20260806 06:31:33.075518 17955 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d65d318d8ff486597dac22338e0bb37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:31:33 UTC) from {username='oryx', principal='oryx/127.7.43.124@KRBTEST.COM'} at 127.0.0.1:45148
I20260806 06:31:33.075562 18245 raft_consensus.cc:2229] T 2d65d318d8ff486597dac22338e0bb37 P 17fc103e74544c2a98d06b27aabb9435 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:31:33.075623 18245 raft_consensus.cc:2258] T 2d65d318d8ff486597dac22338e0bb37 P 17fc103e74544c2a98d06b27aabb9435 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:31:33.075716 17745 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d65d318d8ff486597dac22338e0bb37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:31:33 UTC) from {username='oryx', principal='oryx/127.7.43.124@KRBTEST.COM'} at 127.0.0.1:55356
I20260806 06:31:33.075933 18245 ts_tablet_manager.cc:1905] T 2d65d318d8ff486597dac22338e0bb37 P 17fc103e74544c2a98d06b27aabb9435: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:31:33.076026 18246 tablet_replica.cc:330] T 2d65d318d8ff486597dac22338e0bb37 P 930504b65be34a37ac7455cda9c9c7b6: stopping tablet replica
I20260806 06:31:33.076088 18246 raft_consensus.cc:2229] T 2d65d318d8ff486597dac22338e0bb37 P 930504b65be34a37ac7455cda9c9c7b6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:31:33.076130 18246 raft_consensus.cc:2258] T 2d65d318d8ff486597dac22338e0bb37 P 930504b65be34a37ac7455cda9c9c7b6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:31:33.076236 18247 tablet_replica.cc:330] T 2d65d318d8ff486597dac22338e0bb37 P 40b11f4fbad9437eb9f11d2da5dd0d8b: stopping tablet replica
I20260806 06:31:33.076267 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 17294
I20260806 06:31:33.076289 18247 raft_consensus.cc:2229] T 2d65d318d8ff486597dac22338e0bb37 P 40b11f4fbad9437eb9f11d2da5dd0d8b [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:31:33.093147 18247 raft_consensus.cc:2258] T 2d65d318d8ff486597dac22338e0bb37 P 40b11f4fbad9437eb9f11d2da5dd0d8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:31:33.093477 18247 ts_tablet_manager.cc:1905] T 2d65d318d8ff486597dac22338e0bb37 P 40b11f4fbad9437eb9f11d2da5dd0d8b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:31:33.094844 18246 ts_tablet_manager.cc:1905] T 2d65d318d8ff486597dac22338e0bb37 P 930504b65be34a37ac7455cda9c9c7b6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:31:33.094900 18247 ts_tablet_manager.cc:1918] T 2d65d318d8ff486597dac22338e0bb37 P 40b11f4fbad9437eb9f11d2da5dd0d8b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:31:33.094959 18247 log.cc:1196] T 2d65d318d8ff486597dac22338e0bb37 P 40b11f4fbad9437eb9f11d2da5dd0d8b: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-1/wal/wals/2d65d318d8ff486597dac22338e0bb37
I20260806 06:31:33.095153 18247 ts_tablet_manager.cc:1939] T 2d65d318d8ff486597dac22338e0bb37 P 40b11f4fbad9437eb9f11d2da5dd0d8b: Deleting consensus metadata
I20260806 06:31:33.095443 17547 catalog_manager.cc:4841] TS 40b11f4fbad9437eb9f11d2da5dd0d8b (127.7.43.66:44525): tablet 2d65d318d8ff486597dac22338e0bb37 (table test-table [id=a02e3504c3c24574a6e878f0197cbb25]) successfully deleted
I20260806 06:31:33.096611 18246 ts_tablet_manager.cc:1918] T 2d65d318d8ff486597dac22338e0bb37 P 930504b65be34a37ac7455cda9c9c7b6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:31:33.096728 18246 log.cc:1196] T 2d65d318d8ff486597dac22338e0bb37 P 930504b65be34a37ac7455cda9c9c7b6: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785997861828271-7341-0/minicluster-data/ts-2/wal/wals/2d65d318d8ff486597dac22338e0bb37
I20260806 06:31:33.097024 18246 ts_tablet_manager.cc:1939] T 2d65d318d8ff486597dac22338e0bb37 P 930504b65be34a37ac7455cda9c9c7b6: Deleting consensus metadata
I20260806 06:31:33.097366 17547 catalog_manager.cc:4841] TS 930504b65be34a37ac7455cda9c9c7b6 (127.7.43.67:41259): tablet 2d65d318d8ff486597dac22338e0bb37 (table test-table [id=a02e3504c3c24574a6e878f0197cbb25]) successfully deleted
W20260806 06:31:33.098141 18178 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:33.097938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:39903 (local address 127.0.0.1:40342)
0806 06:31:33.097988 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:33.097999 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:33.098050 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:33.098097 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.65:39903: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":30}
I20260806 06:31:33.098194 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 17521
W20260806 06:31:33.098281 17545 catalog_manager.cc:4584] TS 17fc103e74544c2a98d06b27aabb9435 (127.7.43.65:39903): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2d65d318d8ff486597dac22338e0bb37: Network error: Client connection negotiation failed: client connection to 127.7.43.65:39903: BlockingWrite error: write error: Connection reset by peer (error 104)
I20260806 06:31:33.106455 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 17853
I20260806 06:31:33.117694 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 16848
I20260806 06:31:33.125846 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 17152
2026-08-06T06:31:33Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5314 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:31:33 dist-test-slave-6zbq krb5kdc[18251](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:31:33 dist-test-slave-6zbq krb5kdc[18251](info): set up 2 sockets
Aug 06 06:31:33 dist-test-slave-6zbq krb5kdc[18251](info): commencing operation
krb5kdc: starting...
W20260806 06:31:34.236945 7341 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Aug 06 06:31:34 dist-test-slave-6zbq krb5kdc[18251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997894, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:31:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:31:34Z Disabled control of system clock
WARNING: no policy specified for oryx/127.7.43.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.43.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.43.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/krb5kdc/oryx_127.7.43.126.keytab.
Entry for principal oryx/127.7.43.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/krb5kdc/oryx_127.7.43.126.keytab.
I20260806 06:31:34.341711 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:40427
--webserver_interface=127.7.43.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/krb5kdc/oryx_127.7.43.126.keytab
--principal=oryx/127.7.43.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:39555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.43.126:40427
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:31:34.419466 18272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:34.419648 18272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:34.419667 18272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:34.420724 18272 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:31:34.420755 18272 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 06:31:34.420768 18272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:34.420779 18272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:34.420795 18272 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:31:34.420806 18272 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:31:34.422361 18272 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:39555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.43.126:40427
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/krb5kdc/oryx_127.7.43.126.keytab
--principal=oryx/127.7.43.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:40427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.43.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18272
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:34.422539 18272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:34.422704 18272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:34.427014 18278 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:34.427026 18280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:34.427026 18283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:34.427261 18272 server_base.cc:1034] running on GCE node
I20260806 06:31:34.427495 18272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:34.427748 18272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:34.428882 18272 hybrid_clock.cc:648] HybridClock initialized: now 1785997894428869 us; error 29 us; skew 500 ppm
Aug 06 06:31:34 dist-test-slave-6zbq krb5kdc[18251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997894, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.43.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:34.431124 18272 init.cc:377] Logged in from keytab as oryx/127.7.43.126@KRBTEST.COM (short username oryx)
I20260806 06:31:34.431524 18272 webserver.cc:460] Webserver started at http://127.7.43.126:42387/ using document root <none> and password file <none>
I20260806 06:31:34.431715 18272 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:34.431761 18272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:34.431881 18272 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:34.432832 18272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/master-0/data/instance:
uuid: "5c62144213b245f792a7cecb16ffa196"
format_stamp: "Formatted at 2026-08-06 06:31:34 on dist-test-slave-6zbq"
server_key: "322476898f598de6290644d0e6a307a7"
server_key_iv: "bf287ab6e510cbbe60e103439a387ac4"
server_key_version: "encryptionkey@0"
I20260806 06:31:34.433142 18272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/master-0/wal/instance:
uuid: "5c62144213b245f792a7cecb16ffa196"
format_stamp: "Formatted at 2026-08-06 06:31:34 on dist-test-slave-6zbq"
server_key: "322476898f598de6290644d0e6a307a7"
server_key_iv: "bf287ab6e510cbbe60e103439a387ac4"
server_key_version: "encryptionkey@0"
I20260806 06:31:34.434294 18272 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:31:34.434976 18295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:34.435138 18272 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:31:34.435231 18272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "5c62144213b245f792a7cecb16ffa196"
format_stamp: "Formatted at 2026-08-06 06:31:34 on dist-test-slave-6zbq"
server_key: "322476898f598de6290644d0e6a307a7"
server_key_iv: "bf287ab6e510cbbe60e103439a387ac4"
server_key_version: "encryptionkey@0"
I20260806 06:31:34.435284 18272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:34.452517 18272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:34.454025 18272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:34.454121 18272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:34.458122 18272 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.126:40427
I20260806 06:31:34.458189 18383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.126:40427 every 8 connection(s)
I20260806 06:31:34.458492 18272 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:31:34.459091 18384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:34.461138 18384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c62144213b245f792a7cecb16ffa196: Bootstrap starting.
I20260806 06:31:34.461680 18384 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c62144213b245f792a7cecb16ffa196: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:34.461935 18384 log.cc:824] T 00000000000000000000000000000000 P 5c62144213b245f792a7cecb16ffa196: Log is configured to *not* fsync() on all Append() calls
I20260806 06:31:34.462553 18384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c62144213b245f792a7cecb16ffa196: No bootstrap required, opened a new log
I20260806 06:31:34.464449 18384 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c62144213b245f792a7cecb16ffa196 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c62144213b245f792a7cecb16ffa196" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 40427 } }
I20260806 06:31:34.464573 18384 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c62144213b245f792a7cecb16ffa196 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:34.464604 18384 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c62144213b245f792a7cecb16ffa196 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c62144213b245f792a7cecb16ffa196, State: Initialized, Role: FOLLOWER
I20260806 06:31:34.464721 18384 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c62144213b245f792a7cecb16ffa196 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c62144213b245f792a7cecb16ffa196" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 40427 } }
I20260806 06:31:34.464781 18384 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c62144213b245f792a7cecb16ffa196 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:31:34.464810 18384 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c62144213b245f792a7cecb16ffa196 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:31:34.464855 18384 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c62144213b245f792a7cecb16ffa196 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:34.465330 18384 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c62144213b245f792a7cecb16ffa196 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c62144213b245f792a7cecb16ffa196" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 40427 } }
I20260806 06:31:34.465449 18384 leader_election.cc:302] T 00000000000000000000000000000000 P 5c62144213b245f792a7cecb16ffa196 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c62144213b245f792a7cecb16ffa196; no voters:
I20260806 06:31:34.465541 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 18272
I20260806 06:31:34.465607 18384 leader_election.cc:288] T 00000000000000000000000000000000 P 5c62144213b245f792a7cecb16ffa196 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:31:34.465649 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/master-0/wal/instance
I20260806 06:31:34.465652 18389 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c62144213b245f792a7cecb16ffa196 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:31:34.465793 18389 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c62144213b245f792a7cecb16ffa196 [term 1 LEADER]: Becoming Leader. State: Replica: 5c62144213b245f792a7cecb16ffa196, State: Running, Role: LEADER
I20260806 06:31:34.465868 7341 external_mini_cluster.cc:1428] Setting key 180e5ca3a573a7cc032c6efacc892d8d
I20260806 06:31:34.465883 18384 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c62144213b245f792a7cecb16ffa196 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:31:34.465937 18389 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c62144213b245f792a7cecb16ffa196 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c62144213b245f792a7cecb16ffa196" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 40427 } }
I20260806 06:31:34.466292 18390 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c62144213b245f792a7cecb16ffa196 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c62144213b245f792a7cecb16ffa196" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c62144213b245f792a7cecb16ffa196" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 40427 } } }
I20260806 06:31:34.466384 18390 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c62144213b245f792a7cecb16ffa196 [sys.catalog]: This master's current role is: LEADER
I20260806 06:31:34.466598 18399 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:31:34.466610 18393 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c62144213b245f792a7cecb16ffa196 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c62144213b245f792a7cecb16ffa196. Latest consensus state: current_term: 1 leader_uuid: "5c62144213b245f792a7cecb16ffa196" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c62144213b245f792a7cecb16ffa196" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 40427 } } }
I20260806 06:31:34.466995 18393 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c62144213b245f792a7cecb16ffa196 [sys.catalog]: This master's current role is: LEADER
I20260806 06:31:34.467176 18399 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:31:34.469851 18399 catalog_manager.cc:1347] Generated new cluster ID: c64a2d0948b1471ba55ff01612208e65
I20260806 06:31:34.469915 18399 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:31:34 dist-test-slave-6zbq krb5kdc[18251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997894, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM
I20260806 06:31:34.475603 18398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:34.466293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58210 (local address 127.7.43.126:40427)
0806 06:31:34.466475 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:34.466485 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:34.466903 (+ 418us) server_negotiation.cc:407] Connection header received
0806 06:31:34.467116 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:34.467124 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:34.467200 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:34.467305 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:34.468042 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:34.469341 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:34.470174 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:34.470686 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:34.470782 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:34.472828 (+ 2046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:34.472832 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:34.472832 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:34.472843 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:34.474256 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:34.474594 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:34.474597 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:34.474598 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:34.474684 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:34.474859 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:34.474860 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:34.474860 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:34.475081 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:34.475231 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:34.475335 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 06:31:34.475464 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":93,"threads_started":1}
I20260806 06:31:34.477628 18399 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:31:34.478102 18399 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:31:34.481273 18399 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c62144213b245f792a7cecb16ffa196: Generated new TSK 0
I20260806 06:31:34.481422 18399 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.7.43.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.43.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.43.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/krb5kdc/oryx_127.7.43.65.keytab.
Entry for principal oryx/127.7.43.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/krb5kdc/oryx_127.7.43.65.keytab.
I20260806 06:31:34.506394 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.65:0
--local_ip_for_outbound_sockets=127.7.43.65
--webserver_interface=127.7.43.65
--webserver_port=0
--tserver_master_addrs=127.7.43.126:40427
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/krb5kdc/oryx_127.7.43.65.keytab
--principal=oryx/127.7.43.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:39555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:31:34.584765 18423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:34.584945 18423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:34.584976 18423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:34.586061 18423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:34.586102 18423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:34.586129 18423 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 06:31:34.586160 18423 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.65
I20260806 06:31:34.587643 18423 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:39555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/krb5kdc/oryx_127.7.43.65.keytab
--principal=oryx/127.7.43.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.43.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.43.126:40427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18423
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.65
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:34.587862 18423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:34.588056 18423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:34.592646 18433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:34.592684 18436 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:34.592763 18434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:34.592847 18423 server_base.cc:1034] running on GCE node
I20260806 06:31:34.593076 18423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:34.593264 18423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:34.594395 18423 hybrid_clock.cc:648] HybridClock initialized: now 1785997894594385 us; error 26 us; skew 500 ppm
Aug 06 06:31:34 dist-test-slave-6zbq krb5kdc[18251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997894, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.43.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:34.596627 18423 init.cc:377] Logged in from keytab as oryx/127.7.43.65@KRBTEST.COM (short username oryx)
I20260806 06:31:34.597005 18423 webserver.cc:460] Webserver started at http://127.7.43.65:37559/ using document root <none> and password file <none>
I20260806 06:31:34.597206 18423 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:34.597249 18423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:34.597362 18423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:34.598285 18423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-0/data/instance:
uuid: "32fd9da57e7f4036ab7fb5bf91543079"
format_stamp: "Formatted at 2026-08-06 06:31:34 on dist-test-slave-6zbq"
server_key: "a3ff445568d12c9036138beb29b16ddb"
server_key_iv: "26a002e55660c66f8a3c0a0dc42f853e"
server_key_version: "encryptionkey@0"
I20260806 06:31:34.598629 18423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-0/wal/instance:
uuid: "32fd9da57e7f4036ab7fb5bf91543079"
format_stamp: "Formatted at 2026-08-06 06:31:34 on dist-test-slave-6zbq"
server_key: "a3ff445568d12c9036138beb29b16ddb"
server_key_iv: "26a002e55660c66f8a3c0a0dc42f853e"
server_key_version: "encryptionkey@0"
I20260806 06:31:34.599787 18423 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:31:34.600414 18445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:34.600570 18423 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:31:34.600649 18423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-0/wal
uuid: "32fd9da57e7f4036ab7fb5bf91543079"
format_stamp: "Formatted at 2026-08-06 06:31:34 on dist-test-slave-6zbq"
server_key: "a3ff445568d12c9036138beb29b16ddb"
server_key_iv: "26a002e55660c66f8a3c0a0dc42f853e"
server_key_version: "encryptionkey@0"
I20260806 06:31:34.600704 18423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:34.620493 18423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:34.622036 18423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:34.622131 18423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:34.622617 18423 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:31:34.622699 18423 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:34.622732 18423 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:31:34.622750 18423 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:34.628753 18423 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.65:36049
I20260806 06:31:34.628822 18628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.65:36049 every 8 connection(s)
I20260806 06:31:34.629076 18423 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
Aug 06 06:31:34 dist-test-slave-6zbq krb5kdc[18251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997894, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.43.65@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM
I20260806 06:31:34.630882 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 18423
I20260806 06:31:34.630991 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-0/wal/instance
I20260806 06:31:34.631196 7341 external_mini_cluster.cc:1428] Setting key 89d56e7f42fb06ba1c39a1c1039b47f1
I20260806 06:31:34.632056 18398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:34.622965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:51767 (local address 127.7.43.126:40427)
0806 06:31:34.623011 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:34.623013 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:34.623217 (+ 204us) server_negotiation.cc:407] Connection header received
0806 06:31:34.623617 (+ 400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:34.623618 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:34.623632 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:34.623727 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:34.624428 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:34.625156 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:34.626609 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:34.626912 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:34.626990 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:34.629608 (+ 2618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:34.629610 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:34.629610 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:34.629616 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:34.630766 (+ 1150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:34.631203 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:34.631203 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:34.631204 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:34.631258 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:34.631494 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:34.631495 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:34.631495 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:34.631602 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:34.631649 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:34.631930 (+ 281us) server_negotiation.cc:299] Negotiation successful
0806 06:31:34.631995 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:31:34.632270 18457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:34.622956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40427 (local address 127.7.43.65:51767)
0806 06:31:34.623084 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:34.623120 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:34.623536 (+ 416us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:34.623795 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:34.623799 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:34.624049 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:34.624326 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:34.624332 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:34.625251 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:34.625252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:34.626521 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:34.626524 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:34.627009 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:34.627010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:34.627137 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:34.628007 (+ 870us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:34.628017 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:34.629524 (+ 1507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:34.630838 (+ 1314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:34.630846 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:34.630848 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:34.631125 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:34.631326 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:34.631327 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:34.631327 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:34.631440 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:34.631652 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:34.631657 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:34.631799 (+ 142us) client_negotiation.cc:769] Sending connection context
0806 06:31:34.631911 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 06:31:34.632061 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":68,"threads_started":1}
I20260806 06:31:34.632697 18629 heartbeater.cc:347] Connected to a master server at 127.7.43.126:40427
I20260806 06:31:34.632803 18629 heartbeater.cc:464] Registering TS with master...
I20260806 06:31:34.632984 18629 heartbeater.cc:511] Master 127.7.43.126:40427 requested a full tablet report, sending...
I20260806 06:31:34.633502 18317 ts_manager.cc:194] Registered new tserver with Master: 32fd9da57e7f4036ab7fb5bf91543079 (127.7.43.65:36049)
I20260806 06:31:34.634074 18317 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.43.65@KRBTEST.COM'} at 127.7.43.65:51767
WARNING: no policy specified for oryx/127.7.43.66@KRBTEST.COM; defaulting to no policy
I20260806 06:31:34.640995 18398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:34.634455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:43529 (local address 127.7.43.126:40427)
0806 06:31:34.634493 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:34.634494 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:34.636327 (+ 1833us) server_negotiation.cc:407] Connection header received
0806 06:31:34.636363 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:34.636363 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:34.636373 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:34.636412 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:34.636690 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:34.637119 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:34.637864 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:34.638055 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:34.638106 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:34.638939 (+ 833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:34.638940 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:34.638941 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:34.638947 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:34.640072 (+ 1125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:34.640358 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:34.640359 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:34.640359 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:34.640407 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:34.640561 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:34.640562 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:34.640562 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:34.640655 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:34.640713 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:34.640843 (+ 130us) server_negotiation.cc:299] Negotiation successful
0806 06:31:34.640908 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 06:31:34.641000 18637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:34.634500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40427 (local address 127.7.43.65:43529)
0806 06:31:34.636251 (+ 1751us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:34.636261 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:34.636316 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:34.636416 (+ 100us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:34.636417 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:34.636574 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:34.636621 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:34.636622 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:34.637207 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:34.637208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:34.637790 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:34.637791 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:34.638110 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:34.638111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:34.638187 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:34.638590 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:34.638594 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:34.638858 (+ 264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:34.640142 (+ 1284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:34.640143 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:34.640144 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:34.640299 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:34.640457 (+ 158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:34.640458 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:34.640458 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:34.640513 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:34.640714 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:34.640715 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:34.640757 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 06:31:34.640816 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 06:31:34.640908 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1729,"thread_start_us":63,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.43.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.43.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/krb5kdc/oryx_127.7.43.66.keytab.
Entry for principal oryx/127.7.43.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/krb5kdc/oryx_127.7.43.66.keytab.
I20260806 06:31:34.654621 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.66:0
--local_ip_for_outbound_sockets=127.7.43.66
--webserver_interface=127.7.43.66
--webserver_port=0
--tserver_master_addrs=127.7.43.126:40427
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/krb5kdc/oryx_127.7.43.66.keytab
--principal=oryx/127.7.43.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:39555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:31:34.733748 18640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:34.733927 18640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:34.733958 18640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:34.735092 18640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:34.735131 18640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:34.735158 18640 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 06:31:34.735183 18640 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.66
I20260806 06:31:34.736627 18640 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:39555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/krb5kdc/oryx_127.7.43.66.keytab
--principal=oryx/127.7.43.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.43.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.43.126:40427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18640
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.66
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:34.736857 18640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:34.737056 18640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:34.741585 18648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:34.741607 18647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:34.741611 18650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:34.741683 18640 server_base.cc:1034] running on GCE node
I20260806 06:31:34.741987 18640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:34.742231 18640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:34.743366 18640 hybrid_clock.cc:648] HybridClock initialized: now 1785997894743355 us; error 27 us; skew 500 ppm
Aug 06 06:31:34 dist-test-slave-6zbq krb5kdc[18251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997894, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.43.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:34.745621 18640 init.cc:377] Logged in from keytab as oryx/127.7.43.66@KRBTEST.COM (short username oryx)
I20260806 06:31:34.746021 18640 webserver.cc:460] Webserver started at http://127.7.43.66:35157/ using document root <none> and password file <none>
I20260806 06:31:34.746234 18640 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:34.746281 18640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:34.746423 18640 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:34.747372 18640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-1/data/instance:
uuid: "b4feac633da947dda962b210a9a81e12"
format_stamp: "Formatted at 2026-08-06 06:31:34 on dist-test-slave-6zbq"
server_key: "69045a75cb54892b97024b576e4c6fc9"
server_key_iv: "23a14c1c406274948d66a184cc0f10cd"
server_key_version: "encryptionkey@0"
I20260806 06:31:34.747677 18640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-1/wal/instance:
uuid: "b4feac633da947dda962b210a9a81e12"
format_stamp: "Formatted at 2026-08-06 06:31:34 on dist-test-slave-6zbq"
server_key: "69045a75cb54892b97024b576e4c6fc9"
server_key_iv: "23a14c1c406274948d66a184cc0f10cd"
server_key_version: "encryptionkey@0"
I20260806 06:31:34.748890 18640 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 06:31:34.749528 18662 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:34.749672 18640 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:34.749751 18640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-1/wal
uuid: "b4feac633da947dda962b210a9a81e12"
format_stamp: "Formatted at 2026-08-06 06:31:34 on dist-test-slave-6zbq"
server_key: "69045a75cb54892b97024b576e4c6fc9"
server_key_iv: "23a14c1c406274948d66a184cc0f10cd"
server_key_version: "encryptionkey@0"
I20260806 06:31:34.749811 18640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:34.764408 18640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:34.766065 18640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:34.766182 18640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:34.766665 18640 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:31:34.766710 18640 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:34.766738 18640 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:31:34.766759 18640 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 06:31:34 dist-test-slave-6zbq krb5kdc[18251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997894, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.43.66@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM
I20260806 06:31:34.772975 18640 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.66:40585
I20260806 06:31:34.773034 18836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.66:40585 every 8 connection(s)
I20260806 06:31:34.773312 18640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
I20260806 06:31:34.775552 18398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:34.767093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:39293 (local address 127.7.43.126:40427)
0806 06:31:34.767133 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:34.767134 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:34.767348 (+ 214us) server_negotiation.cc:407] Connection header received
0806 06:31:34.767767 (+ 419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:34.767768 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:34.767783 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:34.767866 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:34.768597 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:34.769325 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:34.770781 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:34.771072 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:34.771138 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:34.773298 (+ 2160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:34.773300 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:34.773300 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:34.773309 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:34.774554 (+ 1245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:34.774893 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:34.774893 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:34.774894 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:34.774938 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:34.775101 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:34.775101 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:34.775101 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:34.775204 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:34.775247 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:34.775435 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 06:31:34.775490 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 06:31:34.775699 18673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:34.767098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40427 (local address 127.7.43.66:39293)
0806 06:31:34.767251 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:34.767281 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:34.767680 (+ 399us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:34.767937 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:34.767942 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:34.768221 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:34.768501 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:34.768508 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:34.769417 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:34.769418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:34.770683 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:34.770686 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:34.771186 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:34.771187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:34.771305 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:34.771735 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:34.771742 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:34.773211 (+ 1469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:34.774634 (+ 1423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:34.774640 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:34.774641 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:34.774839 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:34.774979 (+ 140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:34.774980 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:34.774980 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:34.775061 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:34.775245 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:34.775248 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:34.775343 (+ 95us) client_negotiation.cc:769] Sending connection context
0806 06:31:34.775416 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 06:31:34.775521 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":90,"threads_started":1}
I20260806 06:31:34.776146 18838 heartbeater.cc:347] Connected to a master server at 127.7.43.126:40427
I20260806 06:31:34.776226 18838 heartbeater.cc:464] Registering TS with master...
I20260806 06:31:34.776391 18838 heartbeater.cc:511] Master 127.7.43.126:40427 requested a full tablet report, sending...
I20260806 06:31:34.776734 18317 ts_manager.cc:194] Registered new tserver with Master: b4feac633da947dda962b210a9a81e12 (127.7.43.66:40585)
I20260806 06:31:34.777076 18317 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.43.66@KRBTEST.COM'} at 127.7.43.66:39293
I20260806 06:31:34.778731 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 18640
I20260806 06:31:34.778838 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-1/wal/instance
I20260806 06:31:34.779011 7341 external_mini_cluster.cc:1428] Setting key 432e705fe17ea301bd28617d446645e3
I20260806 06:31:34.782054 18847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:34.777061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40427 (local address 127.7.43.66:60307)
0806 06:31:34.777208 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:34.777218 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:34.777270 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:34.777394 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:34.777394 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:34.777581 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:34.777631 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:34.777633 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:34.778205 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:34.778206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:34.778716 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:34.778717 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:34.779096 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:34.779097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:34.779164 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:34.779531 (+ 367us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:34.779536 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:34.779874 (+ 338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:34.781232 (+ 1358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:34.781233 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:34.781233 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:34.781376 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:34.781541 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:34.781542 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:34.781542 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:34.781584 (+ 42us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:34.781849 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:34.781850 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:34.781881 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 06:31:34.781926 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 06:31:34.781990 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":64,"threads_started":1}
I20260806 06:31:34.782088 18398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:34.777119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:60307 (local address 127.7.43.126:40427)
0806 06:31:34.777172 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:34.777174 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:34.777268 (+ 94us) server_negotiation.cc:407] Connection header received
0806 06:31:34.777313 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:34.777314 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:34.777323 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:34.777386 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:34.777704 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:34.778120 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:34.778786 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:34.779025 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:34.779085 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:34.779944 (+ 859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:34.779945 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:34.779945 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:34.779951 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:34.781149 (+ 1198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:34.781435 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:34.781436 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:34.781436 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:34.781490 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:34.781631 (+ 141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:34.781632 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:34.781632 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:34.781797 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:34.781861 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:34.781948 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 06:31:34.782012 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
WARNING: no policy specified for oryx/127.7.43.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.43.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.43.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/krb5kdc/oryx_127.7.43.67.keytab.
Entry for principal oryx/127.7.43.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/krb5kdc/oryx_127.7.43.67.keytab.
I20260806 06:31:34.800377 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.67:0
--local_ip_for_outbound_sockets=127.7.43.67
--webserver_interface=127.7.43.67
--webserver_port=0
--tserver_master_addrs=127.7.43.126:40427
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/krb5kdc/oryx_127.7.43.67.keytab
--principal=oryx/127.7.43.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:39555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:31:34.878098 18851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:34.878271 18851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:34.878293 18851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:34.879391 18851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:34.879429 18851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:34.879448 18851 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 06:31:34.879477 18851 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.67
I20260806 06:31:34.880900 18851 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:39555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/krb5kdc/oryx_127.7.43.67.keytab
--principal=oryx/127.7.43.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.43.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.43.126:40427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18851
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.67
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:34.881107 18851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:34.881299 18851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:34.885761 18860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:34.885776 18856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:34.885855 18851 server_base.cc:1034] running on GCE node
W20260806 06:31:34.885778 18857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:34.886142 18851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:34.886382 18851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:34.887528 18851 hybrid_clock.cc:648] HybridClock initialized: now 1785997894887517 us; error 29 us; skew 500 ppm
Aug 06 06:31:34 dist-test-slave-6zbq krb5kdc[18251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997894, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.43.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:34.889745 18851 init.cc:377] Logged in from keytab as oryx/127.7.43.67@KRBTEST.COM (short username oryx)
I20260806 06:31:34.890123 18851 webserver.cc:460] Webserver started at http://127.7.43.67:45729/ using document root <none> and password file <none>
I20260806 06:31:34.890302 18851 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:34.890377 18851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:34.890496 18851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:34.891407 18851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-2/data/instance:
uuid: "1f0f48ee9ccb47bd8d010429fe8902a0"
format_stamp: "Formatted at 2026-08-06 06:31:34 on dist-test-slave-6zbq"
server_key: "93ca1aa0e95ccc74f4fe200f1b3e2fb5"
server_key_iv: "24024b1073d2ccdfe694c13f08933594"
server_key_version: "encryptionkey@0"
I20260806 06:31:34.891710 18851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-2/wal/instance:
uuid: "1f0f48ee9ccb47bd8d010429fe8902a0"
format_stamp: "Formatted at 2026-08-06 06:31:34 on dist-test-slave-6zbq"
server_key: "93ca1aa0e95ccc74f4fe200f1b3e2fb5"
server_key_iv: "24024b1073d2ccdfe694c13f08933594"
server_key_version: "encryptionkey@0"
I20260806 06:31:34.892858 18851 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:31:34.893463 18873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:34.893613 18851 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:34.893692 18851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-2/wal
uuid: "1f0f48ee9ccb47bd8d010429fe8902a0"
format_stamp: "Formatted at 2026-08-06 06:31:34 on dist-test-slave-6zbq"
server_key: "93ca1aa0e95ccc74f4fe200f1b3e2fb5"
server_key_iv: "24024b1073d2ccdfe694c13f08933594"
server_key_version: "encryptionkey@0"
I20260806 06:31:34.893755 18851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:34.919551 18851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:34.921427 18851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:34.921554 18851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:34.922017 18851 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:31:34.922055 18851 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:34.922082 18851 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:31:34.922101 18851 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 06:31:34 dist-test-slave-6zbq krb5kdc[18251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997894, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.43.67@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM
I20260806 06:31:34.928763 18851 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.67:43555
I20260806 06:31:34.929117 18851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
I20260806 06:31:34.929193 19053 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.67:43555 every 8 connection(s)
I20260806 06:31:34.930670 18398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:34.922404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:49423 (local address 127.7.43.126:40427)
0806 06:31:34.922452 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:34.922454 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:34.922634 (+ 180us) server_negotiation.cc:407] Connection header received
0806 06:31:34.923178 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:34.923180 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:34.923194 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:34.923278 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:34.923892 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:34.924620 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:34.925638 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:34.925835 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:34.925893 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:34.928256 (+ 2363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:34.928258 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:34.928258 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:34.928267 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:34.929640 (+ 1373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:34.929977 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:34.929978 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:34.929978 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:34.930033 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:34.930183 (+ 150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:34.930184 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:34.930184 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:34.930275 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:34.930316 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:34.930542 (+ 226us) server_negotiation.cc:299] Negotiation successful
0806 06:31:34.930607 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 06:31:34.930778 18885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:34.922399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40427 (local address 127.7.43.67:49423)
0806 06:31:34.922525 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:34.922556 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:34.923061 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:34.923366 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:34.923369 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:34.923642 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:34.923813 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:34.923818 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:34.924714 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:34.924715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:34.925537 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:34.925540 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:34.925901 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:34.925903 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:34.926024 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:34.926470 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:34.926477 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:34.928168 (+ 1691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:34.929707 (+ 1539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:34.929710 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:34.929711 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:34.929908 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:34.930074 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:34.930075 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:34.930075 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:34.930151 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:34.930307 (+ 156us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:34.930309 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:34.930450 (+ 141us) client_negotiation.cc:769] Sending connection context
0806 06:31:34.930526 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 06:31:34.930632 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":67,"threads_started":1}
I20260806 06:31:34.931176 19055 heartbeater.cc:347] Connected to a master server at 127.7.43.126:40427
I20260806 06:31:34.931252 19055 heartbeater.cc:464] Registering TS with master...
I20260806 06:31:34.931404 19055 heartbeater.cc:511] Master 127.7.43.126:40427 requested a full tablet report, sending...
I20260806 06:31:34.931734 18319 ts_manager.cc:194] Registered new tserver with Master: 1f0f48ee9ccb47bd8d010429fe8902a0 (127.7.43.67:43555)
I20260806 06:31:34.932212 18319 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.43.67@KRBTEST.COM'} at 127.7.43.67:49423
I20260806 06:31:34.934552 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 18851
I20260806 06:31:34.934652 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-2/wal/instance
I20260806 06:31:34.934808 7341 external_mini_cluster.cc:1428] Setting key b9e0308ac376e65eded40a253114059f
I20260806 06:31:34.935647 7341 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:31:34.937036 18398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:34.932078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:55837 (local address 127.7.43.126:40427)
0806 06:31:34.932124 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:34.932126 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:34.932176 (+ 50us) server_negotiation.cc:407] Connection header received
0806 06:31:34.932269 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:34.932270 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:34.932279 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:34.932331 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:34.932635 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:34.933102 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:34.933802 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:34.933975 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:34.934028 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:34.934837 (+ 809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:34.934838 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:34.934839 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:34.934846 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:34.936112 (+ 1266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:34.936394 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:34.936395 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:34.936395 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:34.936448 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:34.936603 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:34.936604 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:34.936604 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:34.936701 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:34.936753 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:34.936891 (+ 138us) server_negotiation.cc:299] Negotiation successful
0806 06:31:34.936951 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 06:31:34.937049 19063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:34.932008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40427 (local address 127.7.43.67:55837)
0806 06:31:34.932128 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:34.932136 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:34.932189 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:34.932346 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:34.932347 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:34.932511 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:34.932567 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:34.932568 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:34.933188 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:34.933189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:34.933715 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:34.933716 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:34.934032 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:34.934032 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:34.934076 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:34.934445 (+ 369us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:34.934450 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:34.934762 (+ 312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:34.936180 (+ 1418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:34.936181 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:34.936182 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:34.936336 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:34.936499 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:34.936500 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:34.936500 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:34.936555 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:34.936760 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:34.936761 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:34.936803 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 06:31:34.936861 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 06:31:34.936951 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":75,"threads_started":1}
Aug 06 06:31:34 dist-test-slave-6zbq krb5kdc[18251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997894, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 06:31:34 dist-test-slave-6zbq krb5kdc[18251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:31:34 dist-test-slave-6zbq krb5kdc[18251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 06:31:34.955657 18398 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:34.950999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58218 (local address 127.7.43.126:40427)
0806 06:31:34.951048 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:34.951050 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:34.951057 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:31:34.951123 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:34.951124 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:34.951132 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:34.951178 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:34.951691 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:34.952124 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:34.952826 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:34.952997 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:34.953048 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:34.955597 (+ 2549us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58218: BlockingRecv error: recv got EOF from 127.0.0.1:58218 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 06 06:31:34 dist-test-slave-6zbq krb5kdc[18251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997894, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM
I20260806 06:31:34.961824 18398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:34.956332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58234 (local address 127.7.43.126:40427)
0806 06:31:34.956386 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:34.956388 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:34.956398 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:31:34.956457 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:34.956458 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:34.956466 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:34.956509 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:34.956905 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:34.957333 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:34.957950 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:34.958121 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:34.958171 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:34.959933 (+ 1762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:34.959934 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:34.959935 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:34.959941 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:34.960909 (+ 968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:34.961202 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:34.961203 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:34.961203 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:34.961260 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:34.961418 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:34.961418 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:34.961419 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:34.961509 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:34.961588 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:34.961682 (+ 94us) server_negotiation.cc:299] Negotiation successful
0806 06:31:34.961747 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 06:31:34.963160 18319 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58234:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:31:34.964668 18319 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:31:34.970492 19083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:34.966659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.66:40585 (local address 127.0.0.1:56680)
0806 06:31:34.966895 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:34.966933 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:34.966995 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:34.967254 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:34.967257 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:34.967259 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:31:34.967316 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:34.967318 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:34.968317 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:34.968318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:34.969344 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:34.969347 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:34.970181 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:34.970182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:34.970268 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:34.970282 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 06:31:34.970373 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 06:31:34.970428 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":47,"threads_started":1}
I20260806 06:31:34.970515 19089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:34.966856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56680 (local address 127.7.43.66:40585)
0806 06:31:34.967021 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:34.967025 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:34.967036 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:31:34.967075 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:34.967077 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:34.967093 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:34.967167 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:31:34.967398 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:34.968242 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:34.969438 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:34.970110 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:34.970167 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:34.970239 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:34.970380 (+ 141us) server_negotiation.cc:299] Negotiation successful
0806 06:31:34.970420 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":77,"threads_started":1}
I20260806 06:31:34.970755 19085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:34.966662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.67:43555 (local address 127.0.0.1:50386)
0806 06:31:34.967081 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:34.967090 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:34.967154 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:34.967440 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:34.967442 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:34.967444 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:31:34.967495 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:34.967497 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:34.968735 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:34.968736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:34.969581 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:34.969584 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:34.970527 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:34.970528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:34.970592 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:34.970596 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:31:34.970651 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 06:31:34.970688 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":48,"threads_started":1}
I20260806 06:31:34.970991 19087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:34.966741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50386 (local address 127.7.43.67:43555)
0806 06:31:34.967041 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:34.967045 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:34.967158 (+ 113us) server_negotiation.cc:407] Connection header received
0806 06:31:34.967216 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:34.967218 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:34.967279 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:34.967350 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:31:34.967566 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:34.968640 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:34.969662 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:34.970303 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:34.970427 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:34.970500 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:34.970863 (+ 363us) server_negotiation.cc:299] Negotiation successful
0806 06:31:34.970906 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":80,"threads_started":1}
I20260806 06:31:34.971129 18736 tablet_service.cc:1467] Processing CreateTablet for tablet f5879ef01da0408bb76c839ef5505278 (DEFAULT_TABLE table=test-table [id=47030dcafd0840509cc33596908079b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:31:34.971413 19084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:34.966680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:36049 (local address 127.0.0.1:46106)
0806 06:31:34.967084 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:34.967100 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:34.967134 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:34.967529 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:34.967530 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:34.967531 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:31:34.967565 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:34.967567 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:34.968774 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:34.968775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:34.969813 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:34.969816 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:34.971168 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:34.971170 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:34.971246 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:34.971250 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:31:34.971314 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 06:31:34.971351 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":78,"threads_started":1}
I20260806 06:31:34.971460 19091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:34.966900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46106 (local address 127.7.43.65:36049)
0806 06:31:34.967301 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:34.967305 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:34.967314 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:31:34.967348 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:34.967350 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:34.967364 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:34.967467 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:31:34.967643 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:34.968694 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:34.969892 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:34.970998 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:34.971066 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:34.971139 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:34.971326 (+ 187us) server_negotiation.cc:299] Negotiation successful
0806 06:31:34.971365 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":82,"threads_started":1}
I20260806 06:31:34.971520 18736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5879ef01da0408bb76c839ef5505278. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:34.972074 18521 tablet_service.cc:1467] Processing CreateTablet for tablet f5879ef01da0408bb76c839ef5505278 (DEFAULT_TABLE table=test-table [id=47030dcafd0840509cc33596908079b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:31:34.972398 18521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5879ef01da0408bb76c839ef5505278. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:34.975363 19092 tablet_bootstrap.cc:492] T f5879ef01da0408bb76c839ef5505278 P b4feac633da947dda962b210a9a81e12: Bootstrap starting.
I20260806 06:31:34.975905 18944 tablet_service.cc:1467] Processing CreateTablet for tablet f5879ef01da0408bb76c839ef5505278 (DEFAULT_TABLE table=test-table [id=47030dcafd0840509cc33596908079b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:31:34.976135 19093 tablet_bootstrap.cc:492] T f5879ef01da0408bb76c839ef5505278 P 32fd9da57e7f4036ab7fb5bf91543079: Bootstrap starting.
I20260806 06:31:34.976207 19092 tablet_bootstrap.cc:654] T f5879ef01da0408bb76c839ef5505278 P b4feac633da947dda962b210a9a81e12: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:34.976243 18944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5879ef01da0408bb76c839ef5505278. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:34.976526 19092 log.cc:824] T f5879ef01da0408bb76c839ef5505278 P b4feac633da947dda962b210a9a81e12: Log is configured to *not* fsync() on all Append() calls
I20260806 06:31:34.976642 19093 tablet_bootstrap.cc:654] T f5879ef01da0408bb76c839ef5505278 P 32fd9da57e7f4036ab7fb5bf91543079: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:34.976908 19093 log.cc:824] T f5879ef01da0408bb76c839ef5505278 P 32fd9da57e7f4036ab7fb5bf91543079: Log is configured to *not* fsync() on all Append() calls
I20260806 06:31:34.977308 19092 tablet_bootstrap.cc:492] T f5879ef01da0408bb76c839ef5505278 P b4feac633da947dda962b210a9a81e12: No bootstrap required, opened a new log
I20260806 06:31:34.977370 19092 ts_tablet_manager.cc:1397] T f5879ef01da0408bb76c839ef5505278 P b4feac633da947dda962b210a9a81e12: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:31:34.977536 19093 tablet_bootstrap.cc:492] T f5879ef01da0408bb76c839ef5505278 P 32fd9da57e7f4036ab7fb5bf91543079: No bootstrap required, opened a new log
I20260806 06:31:34.977576 19093 ts_tablet_manager.cc:1397] T f5879ef01da0408bb76c839ef5505278 P 32fd9da57e7f4036ab7fb5bf91543079: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:31:34.979079 19093 raft_consensus.cc:348] T f5879ef01da0408bb76c839ef5505278 P 32fd9da57e7f4036ab7fb5bf91543079 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } } peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } }
I20260806 06:31:34.979204 19093 raft_consensus.cc:374] T f5879ef01da0408bb76c839ef5505278 P 32fd9da57e7f4036ab7fb5bf91543079 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:34.979249 19093 raft_consensus.cc:729] T f5879ef01da0408bb76c839ef5505278 P 32fd9da57e7f4036ab7fb5bf91543079 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32fd9da57e7f4036ab7fb5bf91543079, State: Initialized, Role: FOLLOWER
I20260806 06:31:34.979363 19093 consensus_queue.cc:260] T f5879ef01da0408bb76c839ef5505278 P 32fd9da57e7f4036ab7fb5bf91543079 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } } peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } }
I20260806 06:31:34.979326 19092 raft_consensus.cc:348] T f5879ef01da0408bb76c839ef5505278 P b4feac633da947dda962b210a9a81e12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } } peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } }
I20260806 06:31:34.979485 19092 raft_consensus.cc:374] T f5879ef01da0408bb76c839ef5505278 P b4feac633da947dda962b210a9a81e12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:34.979532 19092 raft_consensus.cc:729] T f5879ef01da0408bb76c839ef5505278 P b4feac633da947dda962b210a9a81e12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4feac633da947dda962b210a9a81e12, State: Initialized, Role: FOLLOWER
I20260806 06:31:34.979630 19093 ts_tablet_manager.cc:1428] T f5879ef01da0408bb76c839ef5505278 P 32fd9da57e7f4036ab7fb5bf91543079: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:31:34.979624 19097 tablet_bootstrap.cc:492] T f5879ef01da0408bb76c839ef5505278 P 1f0f48ee9ccb47bd8d010429fe8902a0: Bootstrap starting.
I20260806 06:31:34.979662 19092 consensus_queue.cc:260] T f5879ef01da0408bb76c839ef5505278 P b4feac633da947dda962b210a9a81e12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } } peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } }
I20260806 06:31:34.979902 18629 heartbeater.cc:503] Master 127.7.43.126:40427 was elected leader, sending a full tablet report...
I20260806 06:31:34.979913 19092 ts_tablet_manager.cc:1428] T f5879ef01da0408bb76c839ef5505278 P b4feac633da947dda962b210a9a81e12: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:31:34.979959 18838 heartbeater.cc:503] Master 127.7.43.126:40427 was elected leader, sending a full tablet report...
I20260806 06:31:34.980296 19097 tablet_bootstrap.cc:654] T f5879ef01da0408bb76c839ef5505278 P 1f0f48ee9ccb47bd8d010429fe8902a0: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:34.980602 19097 log.cc:824] T f5879ef01da0408bb76c839ef5505278 P 1f0f48ee9ccb47bd8d010429fe8902a0: Log is configured to *not* fsync() on all Append() calls
I20260806 06:31:34.981341 19097 tablet_bootstrap.cc:492] T f5879ef01da0408bb76c839ef5505278 P 1f0f48ee9ccb47bd8d010429fe8902a0: No bootstrap required, opened a new log
I20260806 06:31:34.981400 19097 ts_tablet_manager.cc:1397] T f5879ef01da0408bb76c839ef5505278 P 1f0f48ee9ccb47bd8d010429fe8902a0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:31:34.982915 19097 raft_consensus.cc:348] T f5879ef01da0408bb76c839ef5505278 P 1f0f48ee9ccb47bd8d010429fe8902a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } } peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } }
I20260806 06:31:34.983039 19097 raft_consensus.cc:374] T f5879ef01da0408bb76c839ef5505278 P 1f0f48ee9ccb47bd8d010429fe8902a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:34.983069 19097 raft_consensus.cc:729] T f5879ef01da0408bb76c839ef5505278 P 1f0f48ee9ccb47bd8d010429fe8902a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f0f48ee9ccb47bd8d010429fe8902a0, State: Initialized, Role: FOLLOWER
I20260806 06:31:34.983172 19097 consensus_queue.cc:260] T f5879ef01da0408bb76c839ef5505278 P 1f0f48ee9ccb47bd8d010429fe8902a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } } peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } }
I20260806 06:31:34.983383 19097 ts_tablet_manager.cc:1428] T f5879ef01da0408bb76c839ef5505278 P 1f0f48ee9ccb47bd8d010429fe8902a0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:31:34.983426 19055 heartbeater.cc:503] Master 127.7.43.126:40427 was elected leader, sending a full tablet report...
W20260806 06:31:35.023887 18840 tablet.cc:2369] T f5879ef01da0408bb76c839ef5505278 P b4feac633da947dda962b210a9a81e12: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 06:31:35.129774 18631 tablet.cc:2369] T f5879ef01da0408bb76c839ef5505278 P 32fd9da57e7f4036ab7fb5bf91543079: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 06:31:35.179615 19058 tablet.cc:2369] T f5879ef01da0408bb76c839ef5505278 P 1f0f48ee9ccb47bd8d010429fe8902a0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:31:35.258172 19098 raft_consensus.cc:482] T f5879ef01da0408bb76c839ef5505278 P 32fd9da57e7f4036ab7fb5bf91543079 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:31:35.258312 19098 raft_consensus.cc:504] T f5879ef01da0408bb76c839ef5505278 P 32fd9da57e7f4036ab7fb5bf91543079 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } } peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } }
I20260806 06:31:35.258700 19098 leader_election.cc:288] T f5879ef01da0408bb76c839ef5505278 P 32fd9da57e7f4036ab7fb5bf91543079 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f0f48ee9ccb47bd8d010429fe8902a0 (127.7.43.67:43555), b4feac633da947dda962b210a9a81e12 (127.7.43.66:40585)
I20260806 06:31:35.261440 19107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:35.258854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.67:43555 (local address 127.7.43.65:33115)
0806 06:31:35.258977 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:35.258987 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:35.259048 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:35.259188 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:35.259189 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:35.259192 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:31:35.259274 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:35.259276 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:35.259904 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:35.259905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:35.260710 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:35.260713 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:35.261247 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:35.261248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:35.261307 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:35.261310 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:31:35.261348 (+ 38us) client_negotiation.cc:240] Negotiation successful
0806 06:31:35.261375 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"mutex_wait_us":39,"thread_start_us":65,"threads_started":1}
I20260806 06:31:35.261438 19087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:35.258933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:33115 (local address 127.7.43.67:43555)
0806 06:31:35.258976 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:35.258978 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:35.259050 (+ 72us) server_negotiation.cc:407] Connection header received
0806 06:31:35.259103 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:35.259105 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:35.259117 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:35.259175 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:31:35.259339 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:35.259828 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:35.260803 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:35.261168 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:35.261231 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:35.261239 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:35.261354 (+ 115us) server_negotiation.cc:299] Negotiation successful
0806 06:31:35.261380 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 06:31:35.261746 18978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5879ef01da0408bb76c839ef5505278" candidate_uuid: "32fd9da57e7f4036ab7fb5bf91543079" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" is_pre_election: true
I20260806 06:31:35.261808 19103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:35.258854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.66:40585 (local address 127.7.43.65:40045)
0806 06:31:35.259042 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:35.259054 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:35.259138 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:35.259271 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:35.259272 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:35.259274 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:31:35.259340 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:35.259342 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:35.259926 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:35.259928 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:35.260694 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:35.260696 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:35.261561 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:35.261562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:35.261638 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:35.261643 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:31:35.261697 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 06:31:35.261735 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":14,"thread_start_us":55,"threads_started":1}
I20260806 06:31:35.261816 19089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:35.258933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:40045 (local address 127.7.43.66:40585)
0806 06:31:35.258982 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:35.258984 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:35.259136 (+ 152us) server_negotiation.cc:407] Connection header received
0806 06:31:35.259196 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:35.259197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:35.259209 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:35.259263 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:31:35.259397 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:35.259853 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:35.260772 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:35.261137 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:35.261195 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:35.261205 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:35.261726 (+ 521us) server_negotiation.cc:299] Negotiation successful
0806 06:31:35.261764 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:31:35.261904 18978 raft_consensus.cc:2454] T f5879ef01da0408bb76c839ef5505278 P 1f0f48ee9ccb47bd8d010429fe8902a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32fd9da57e7f4036ab7fb5bf91543079 in term 0.
I20260806 06:31:35.262104 18448 leader_election.cc:302] T f5879ef01da0408bb76c839ef5505278 P 32fd9da57e7f4036ab7fb5bf91543079 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f0f48ee9ccb47bd8d010429fe8902a0, 32fd9da57e7f4036ab7fb5bf91543079; no voters:
I20260806 06:31:35.262151 18764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5879ef01da0408bb76c839ef5505278" candidate_uuid: "32fd9da57e7f4036ab7fb5bf91543079" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4feac633da947dda962b210a9a81e12" is_pre_election: true
I20260806 06:31:35.262225 19098 raft_consensus.cc:2781] T f5879ef01da0408bb76c839ef5505278 P 32fd9da57e7f4036ab7fb5bf91543079 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:31:35.262281 19098 raft_consensus.cc:482] T f5879ef01da0408bb76c839ef5505278 P 32fd9da57e7f4036ab7fb5bf91543079 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:31:35.262301 19098 raft_consensus.cc:3037] T f5879ef01da0408bb76c839ef5505278 P 32fd9da57e7f4036ab7fb5bf91543079 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:35.262298 18764 raft_consensus.cc:2454] T f5879ef01da0408bb76c839ef5505278 P b4feac633da947dda962b210a9a81e12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32fd9da57e7f4036ab7fb5bf91543079 in term 0.
I20260806 06:31:35.263017 19098 raft_consensus.cc:504] T f5879ef01da0408bb76c839ef5505278 P 32fd9da57e7f4036ab7fb5bf91543079 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } } peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } }
I20260806 06:31:35.263127 19098 leader_election.cc:288] T f5879ef01da0408bb76c839ef5505278 P 32fd9da57e7f4036ab7fb5bf91543079 [CANDIDATE]: Term 1 election: Requested vote from peers 1f0f48ee9ccb47bd8d010429fe8902a0 (127.7.43.67:43555), b4feac633da947dda962b210a9a81e12 (127.7.43.66:40585)
I20260806 06:31:35.263278 18764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5879ef01da0408bb76c839ef5505278" candidate_uuid: "32fd9da57e7f4036ab7fb5bf91543079" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4feac633da947dda962b210a9a81e12"
I20260806 06:31:35.263298 18978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5879ef01da0408bb76c839ef5505278" candidate_uuid: "32fd9da57e7f4036ab7fb5bf91543079" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0"
I20260806 06:31:35.263353 18764 raft_consensus.cc:3037] T f5879ef01da0408bb76c839ef5505278 P b4feac633da947dda962b210a9a81e12 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:35.263366 18978 raft_consensus.cc:3037] T f5879ef01da0408bb76c839ef5505278 P 1f0f48ee9ccb47bd8d010429fe8902a0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:35.264020 18978 raft_consensus.cc:2454] T f5879ef01da0408bb76c839ef5505278 P 1f0f48ee9ccb47bd8d010429fe8902a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32fd9da57e7f4036ab7fb5bf91543079 in term 1.
I20260806 06:31:35.264096 18764 raft_consensus.cc:2454] T f5879ef01da0408bb76c839ef5505278 P b4feac633da947dda962b210a9a81e12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32fd9da57e7f4036ab7fb5bf91543079 in term 1.
I20260806 06:31:35.264175 18448 leader_election.cc:302] T f5879ef01da0408bb76c839ef5505278 P 32fd9da57e7f4036ab7fb5bf91543079 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f0f48ee9ccb47bd8d010429fe8902a0, 32fd9da57e7f4036ab7fb5bf91543079; no voters:
I20260806 06:31:35.264245 19098 raft_consensus.cc:2781] T f5879ef01da0408bb76c839ef5505278 P 32fd9da57e7f4036ab7fb5bf91543079 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:31:35.264346 19098 raft_consensus.cc:686] T f5879ef01da0408bb76c839ef5505278 P 32fd9da57e7f4036ab7fb5bf91543079 [term 1 LEADER]: Becoming Leader. State: Replica: 32fd9da57e7f4036ab7fb5bf91543079, State: Running, Role: LEADER
I20260806 06:31:35.264436 19098 consensus_queue.cc:237] T f5879ef01da0408bb76c839ef5505278 P 32fd9da57e7f4036ab7fb5bf91543079 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } } peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } }
I20260806 06:31:35.265172 18319 catalog_manager.cc:5495] T f5879ef01da0408bb76c839ef5505278 P 32fd9da57e7f4036ab7fb5bf91543079 reported cstate change: term changed from 0 to 1, leader changed from <none> to 32fd9da57e7f4036ab7fb5bf91543079 (127.7.43.65). New cstate: current_term: 1 leader_uuid: "32fd9da57e7f4036ab7fb5bf91543079" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } health_report { overall_health: HEALTHY } } }
I20260806 06:31:35.319854 19091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:35.317624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46108 (local address 127.7.43.65:36049)
0806 06:31:35.317680 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:35.317683 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:35.317692 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:31:35.317757 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:35.317758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:35.317778 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:35.317837 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:31:35.317957 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:35.318467 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:35.319183 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:35.319365 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:35.319419 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:35.319568 (+ 149us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:31:35.319663 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:31:35.319730 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:35.319765 (+ 35us) server_negotiation.cc:299] Negotiation successful
0806 06:31:35.319784 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 06:31:35.321560 18978 raft_consensus.cc:1261] T f5879ef01da0408bb76c839ef5505278 P 1f0f48ee9ccb47bd8d010429fe8902a0 [term 1 FOLLOWER]: Refusing update from remote peer 32fd9da57e7f4036ab7fb5bf91543079: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:31:35.321560 18764 raft_consensus.cc:1261] T f5879ef01da0408bb76c839ef5505278 P b4feac633da947dda962b210a9a81e12 [term 1 FOLLOWER]: Refusing update from remote peer 32fd9da57e7f4036ab7fb5bf91543079: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:31:35.321902 19098 consensus_queue.cc:1035] T f5879ef01da0408bb76c839ef5505278 P 32fd9da57e7f4036ab7fb5bf91543079 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:31:35.322054 19109 consensus_queue.cc:1035] T f5879ef01da0408bb76c839ef5505278 P 32fd9da57e7f4036ab7fb5bf91543079 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:31:35.326768 18317 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58234:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:31:35.326862 18317 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58234:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:31:35.327559 18317 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5c62144213b245f792a7cecb16ffa196: Sending DeleteTablet for 3 replicas of tablet f5879ef01da0408bb76c839ef5505278
I20260806 06:31:35.327886 18521 tablet_service.cc:1514] Processing DeleteTablet for tablet f5879ef01da0408bb76c839ef5505278 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:31:35 UTC) from {username='oryx', principal='oryx/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:46106
I20260806 06:31:35.328016 18944 tablet_service.cc:1514] Processing DeleteTablet for tablet f5879ef01da0408bb76c839ef5505278 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:31:35 UTC) from {username='oryx', principal='oryx/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:50386
I20260806 06:31:35.328107 19126 tablet_replica.cc:330] T f5879ef01da0408bb76c839ef5505278 P 32fd9da57e7f4036ab7fb5bf91543079: stopping tablet replica
I20260806 06:31:35.328090 18736 tablet_service.cc:1514] Processing DeleteTablet for tablet f5879ef01da0408bb76c839ef5505278 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:31:35 UTC) from {username='oryx', principal='oryx/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:56680
I20260806 06:31:35.328193 19126 raft_consensus.cc:2229] T f5879ef01da0408bb76c839ef5505278 P 32fd9da57e7f4036ab7fb5bf91543079 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:31:35.328342 19126 raft_consensus.cc:2258] T f5879ef01da0408bb76c839ef5505278 P 32fd9da57e7f4036ab7fb5bf91543079 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:31:35.328433 19128 tablet_replica.cc:330] T f5879ef01da0408bb76c839ef5505278 P b4feac633da947dda962b210a9a81e12: stopping tablet replica
I20260806 06:31:35.328480 18317 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58234:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 06:31:35.328573 19128 raft_consensus.cc:2229] T f5879ef01da0408bb76c839ef5505278 P b4feac633da947dda962b210a9a81e12 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260806 06:31:35.328626 18317 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:31:35.328671 19128 raft_consensus.cc:2258] T f5879ef01da0408bb76c839ef5505278 P b4feac633da947dda962b210a9a81e12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:31:35.329025 19126 ts_tablet_manager.cc:1905] T f5879ef01da0408bb76c839ef5505278 P 32fd9da57e7f4036ab7fb5bf91543079: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:31:35.329044 19128 ts_tablet_manager.cc:1905] T f5879ef01da0408bb76c839ef5505278 P b4feac633da947dda962b210a9a81e12: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:31:35.330220 18521 tablet_service.cc:1467] Processing CreateTablet for tablet 4a2744552e5341c1bcec227f2f457d2e (DEFAULT_TABLE table=test-table [id=818508665f4a480880538da05ec188ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:31:35.330360 18521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a2744552e5341c1bcec227f2f457d2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:35.330693 19128 ts_tablet_manager.cc:1918] T f5879ef01da0408bb76c839ef5505278 P b4feac633da947dda962b210a9a81e12: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:31:35.330732 19126 ts_tablet_manager.cc:1918] T f5879ef01da0408bb76c839ef5505278 P 32fd9da57e7f4036ab7fb5bf91543079: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:31:35.330757 19128 log.cc:1196] T f5879ef01da0408bb76c839ef5505278 P b4feac633da947dda962b210a9a81e12: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-1/wal/wals/f5879ef01da0408bb76c839ef5505278
I20260806 06:31:35.330787 19126 log.cc:1196] T f5879ef01da0408bb76c839ef5505278 P 32fd9da57e7f4036ab7fb5bf91543079: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-0/wal/wals/f5879ef01da0408bb76c839ef5505278
I20260806 06:31:35.330988 19128 ts_tablet_manager.cc:1939] T f5879ef01da0408bb76c839ef5505278 P b4feac633da947dda962b210a9a81e12: Deleting consensus metadata
I20260806 06:31:35.330989 19126 ts_tablet_manager.cc:1939] T f5879ef01da0408bb76c839ef5505278 P 32fd9da57e7f4036ab7fb5bf91543079: Deleting consensus metadata
I20260806 06:31:35.331586 19093 tablet_bootstrap.cc:492] T 4a2744552e5341c1bcec227f2f457d2e P 32fd9da57e7f4036ab7fb5bf91543079: Bootstrap starting.
I20260806 06:31:35.331895 19093 tablet_bootstrap.cc:654] T 4a2744552e5341c1bcec227f2f457d2e P 32fd9da57e7f4036ab7fb5bf91543079: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:35.331992 18944 tablet_service.cc:1467] Processing CreateTablet for tablet 4a2744552e5341c1bcec227f2f457d2e (DEFAULT_TABLE table=test-table [id=818508665f4a480880538da05ec188ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:31:35.332101 18944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a2744552e5341c1bcec227f2f457d2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:35.332427 19093 tablet_bootstrap.cc:492] T 4a2744552e5341c1bcec227f2f457d2e P 32fd9da57e7f4036ab7fb5bf91543079: No bootstrap required, opened a new log
I20260806 06:31:35.332465 19093 ts_tablet_manager.cc:1397] T 4a2744552e5341c1bcec227f2f457d2e P 32fd9da57e7f4036ab7fb5bf91543079: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:31:35.332572 19093 raft_consensus.cc:348] T 4a2744552e5341c1bcec227f2f457d2e P 32fd9da57e7f4036ab7fb5bf91543079 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } } peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } }
I20260806 06:31:35.332623 19093 raft_consensus.cc:374] T 4a2744552e5341c1bcec227f2f457d2e P 32fd9da57e7f4036ab7fb5bf91543079 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:35.332639 19093 raft_consensus.cc:729] T 4a2744552e5341c1bcec227f2f457d2e P 32fd9da57e7f4036ab7fb5bf91543079 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32fd9da57e7f4036ab7fb5bf91543079, State: Initialized, Role: FOLLOWER
I20260806 06:31:35.332679 19093 consensus_queue.cc:260] T 4a2744552e5341c1bcec227f2f457d2e P 32fd9da57e7f4036ab7fb5bf91543079 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } } peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } }
I20260806 06:31:35.332758 19093 ts_tablet_manager.cc:1428] T 4a2744552e5341c1bcec227f2f457d2e P 32fd9da57e7f4036ab7fb5bf91543079: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:35.333384 19097 tablet_bootstrap.cc:492] T 4a2744552e5341c1bcec227f2f457d2e P 1f0f48ee9ccb47bd8d010429fe8902a0: Bootstrap starting.
I20260806 06:31:35.333783 19097 tablet_bootstrap.cc:654] T 4a2744552e5341c1bcec227f2f457d2e P 1f0f48ee9ccb47bd8d010429fe8902a0: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:35.333791 18300 catalog_manager.cc:4841] TS 32fd9da57e7f4036ab7fb5bf91543079 (127.7.43.65:36049): tablet f5879ef01da0408bb76c839ef5505278 (table test-table [id=47030dcafd0840509cc33596908079b5]) successfully deleted
I20260806 06:31:35.334192 18298 catalog_manager.cc:4841] TS b4feac633da947dda962b210a9a81e12 (127.7.43.66:40585): tablet f5879ef01da0408bb76c839ef5505278 (table test-table [id=47030dcafd0840509cc33596908079b5]) successfully deleted
I20260806 06:31:35.334296 19097 tablet_bootstrap.cc:492] T 4a2744552e5341c1bcec227f2f457d2e P 1f0f48ee9ccb47bd8d010429fe8902a0: No bootstrap required, opened a new log
I20260806 06:31:35.334363 19097 ts_tablet_manager.cc:1397] T 4a2744552e5341c1bcec227f2f457d2e P 1f0f48ee9ccb47bd8d010429fe8902a0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:31:35.334386 18736 tablet_service.cc:1467] Processing CreateTablet for tablet 4a2744552e5341c1bcec227f2f457d2e (DEFAULT_TABLE table=test-table [id=818508665f4a480880538da05ec188ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:31:35.334478 18736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a2744552e5341c1bcec227f2f457d2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:35.334506 19097 raft_consensus.cc:348] T 4a2744552e5341c1bcec227f2f457d2e P 1f0f48ee9ccb47bd8d010429fe8902a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } } peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } }
I20260806 06:31:35.334563 19097 raft_consensus.cc:374] T 4a2744552e5341c1bcec227f2f457d2e P 1f0f48ee9ccb47bd8d010429fe8902a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:35.328339 19127 tablet_replica.cc:330] T f5879ef01da0408bb76c839ef5505278 P 1f0f48ee9ccb47bd8d010429fe8902a0: stopping tablet replica
I20260806 06:31:35.334581 19097 raft_consensus.cc:729] T 4a2744552e5341c1bcec227f2f457d2e P 1f0f48ee9ccb47bd8d010429fe8902a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f0f48ee9ccb47bd8d010429fe8902a0, State: Initialized, Role: FOLLOWER
I20260806 06:31:35.334627 19097 consensus_queue.cc:260] T 4a2744552e5341c1bcec227f2f457d2e P 1f0f48ee9ccb47bd8d010429fe8902a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } } peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } }
I20260806 06:31:35.334723 19097 ts_tablet_manager.cc:1428] T 4a2744552e5341c1bcec227f2f457d2e P 1f0f48ee9ccb47bd8d010429fe8902a0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:35.335417 19092 tablet_bootstrap.cc:492] T 4a2744552e5341c1bcec227f2f457d2e P b4feac633da947dda962b210a9a81e12: Bootstrap starting.
I20260806 06:31:35.335382 19127 raft_consensus.cc:2229] T f5879ef01da0408bb76c839ef5505278 P 1f0f48ee9ccb47bd8d010429fe8902a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:31:35.335572 19127 raft_consensus.cc:2258] T f5879ef01da0408bb76c839ef5505278 P 1f0f48ee9ccb47bd8d010429fe8902a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:31:35.335736 19092 tablet_bootstrap.cc:654] T 4a2744552e5341c1bcec227f2f457d2e P b4feac633da947dda962b210a9a81e12: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:35.336249 19092 tablet_bootstrap.cc:492] T 4a2744552e5341c1bcec227f2f457d2e P b4feac633da947dda962b210a9a81e12: No bootstrap required, opened a new log
I20260806 06:31:35.336290 19092 ts_tablet_manager.cc:1397] T 4a2744552e5341c1bcec227f2f457d2e P b4feac633da947dda962b210a9a81e12: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:31:35.336297 19127 ts_tablet_manager.cc:1905] T f5879ef01da0408bb76c839ef5505278 P 1f0f48ee9ccb47bd8d010429fe8902a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:31:35.336566 19092 raft_consensus.cc:348] T 4a2744552e5341c1bcec227f2f457d2e P b4feac633da947dda962b210a9a81e12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } } peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } }
I20260806 06:31:35.336625 19092 raft_consensus.cc:374] T 4a2744552e5341c1bcec227f2f457d2e P b4feac633da947dda962b210a9a81e12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:35.336643 19092 raft_consensus.cc:729] T 4a2744552e5341c1bcec227f2f457d2e P b4feac633da947dda962b210a9a81e12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4feac633da947dda962b210a9a81e12, State: Initialized, Role: FOLLOWER
I20260806 06:31:35.336678 19092 consensus_queue.cc:260] T 4a2744552e5341c1bcec227f2f457d2e P b4feac633da947dda962b210a9a81e12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } } peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } }
I20260806 06:31:35.336776 19092 ts_tablet_manager.cc:1428] T 4a2744552e5341c1bcec227f2f457d2e P b4feac633da947dda962b210a9a81e12: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 06:31:35.337831 19127 ts_tablet_manager.cc:1918] T f5879ef01da0408bb76c839ef5505278 P 1f0f48ee9ccb47bd8d010429fe8902a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:31:35.337981 19127 log.cc:1196] T f5879ef01da0408bb76c839ef5505278 P 1f0f48ee9ccb47bd8d010429fe8902a0: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-2/wal/wals/f5879ef01da0408bb76c839ef5505278
I20260806 06:31:35.338279 19127 ts_tablet_manager.cc:1939] T f5879ef01da0408bb76c839ef5505278 P 1f0f48ee9ccb47bd8d010429fe8902a0: Deleting consensus metadata
I20260806 06:31:35.338654 18296 catalog_manager.cc:4841] TS 1f0f48ee9ccb47bd8d010429fe8902a0 (127.7.43.67:43555): tablet f5879ef01da0408bb76c839ef5505278 (table test-table [id=47030dcafd0840509cc33596908079b5]) successfully deleted
I20260806 06:31:35.397989 19122 raft_consensus.cc:482] T 4a2744552e5341c1bcec227f2f457d2e P 32fd9da57e7f4036ab7fb5bf91543079 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:31:35.398080 19122 raft_consensus.cc:504] T 4a2744552e5341c1bcec227f2f457d2e P 32fd9da57e7f4036ab7fb5bf91543079 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } } peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } }
I20260806 06:31:35.398223 19122 leader_election.cc:288] T 4a2744552e5341c1bcec227f2f457d2e P 32fd9da57e7f4036ab7fb5bf91543079 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4feac633da947dda962b210a9a81e12 (127.7.43.66:40585), 1f0f48ee9ccb47bd8d010429fe8902a0 (127.7.43.67:43555)
I20260806 06:31:35.398447 18978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a2744552e5341c1bcec227f2f457d2e" candidate_uuid: "32fd9da57e7f4036ab7fb5bf91543079" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" is_pre_election: true
I20260806 06:31:35.398463 18764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a2744552e5341c1bcec227f2f457d2e" candidate_uuid: "32fd9da57e7f4036ab7fb5bf91543079" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4feac633da947dda962b210a9a81e12" is_pre_election: true
I20260806 06:31:35.398552 18978 raft_consensus.cc:2454] T 4a2744552e5341c1bcec227f2f457d2e P 1f0f48ee9ccb47bd8d010429fe8902a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32fd9da57e7f4036ab7fb5bf91543079 in term 0.
I20260806 06:31:35.398552 18764 raft_consensus.cc:2454] T 4a2744552e5341c1bcec227f2f457d2e P b4feac633da947dda962b210a9a81e12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32fd9da57e7f4036ab7fb5bf91543079 in term 0.
I20260806 06:31:35.398697 18448 leader_election.cc:302] T 4a2744552e5341c1bcec227f2f457d2e P 32fd9da57e7f4036ab7fb5bf91543079 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f0f48ee9ccb47bd8d010429fe8902a0, 32fd9da57e7f4036ab7fb5bf91543079; no voters:
I20260806 06:31:35.398793 19122 raft_consensus.cc:2781] T 4a2744552e5341c1bcec227f2f457d2e P 32fd9da57e7f4036ab7fb5bf91543079 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:31:35.398833 19122 raft_consensus.cc:482] T 4a2744552e5341c1bcec227f2f457d2e P 32fd9da57e7f4036ab7fb5bf91543079 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:31:35.398847 19122 raft_consensus.cc:3037] T 4a2744552e5341c1bcec227f2f457d2e P 32fd9da57e7f4036ab7fb5bf91543079 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:35.399533 19122 raft_consensus.cc:504] T 4a2744552e5341c1bcec227f2f457d2e P 32fd9da57e7f4036ab7fb5bf91543079 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } } peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } }
I20260806 06:31:35.399653 19122 leader_election.cc:288] T 4a2744552e5341c1bcec227f2f457d2e P 32fd9da57e7f4036ab7fb5bf91543079 [CANDIDATE]: Term 1 election: Requested vote from peers b4feac633da947dda962b210a9a81e12 (127.7.43.66:40585), 1f0f48ee9ccb47bd8d010429fe8902a0 (127.7.43.67:43555)
I20260806 06:31:35.399817 18978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a2744552e5341c1bcec227f2f457d2e" candidate_uuid: "32fd9da57e7f4036ab7fb5bf91543079" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0"
I20260806 06:31:35.399824 18764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a2744552e5341c1bcec227f2f457d2e" candidate_uuid: "32fd9da57e7f4036ab7fb5bf91543079" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4feac633da947dda962b210a9a81e12"
I20260806 06:31:35.399886 18978 raft_consensus.cc:3037] T 4a2744552e5341c1bcec227f2f457d2e P 1f0f48ee9ccb47bd8d010429fe8902a0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:35.399909 18764 raft_consensus.cc:3037] T 4a2744552e5341c1bcec227f2f457d2e P b4feac633da947dda962b210a9a81e12 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:35.400489 18978 raft_consensus.cc:2454] T 4a2744552e5341c1bcec227f2f457d2e P 1f0f48ee9ccb47bd8d010429fe8902a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32fd9da57e7f4036ab7fb5bf91543079 in term 1.
I20260806 06:31:35.400661 18448 leader_election.cc:302] T 4a2744552e5341c1bcec227f2f457d2e P 32fd9da57e7f4036ab7fb5bf91543079 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f0f48ee9ccb47bd8d010429fe8902a0, 32fd9da57e7f4036ab7fb5bf91543079; no voters:
I20260806 06:31:35.400741 19122 raft_consensus.cc:2781] T 4a2744552e5341c1bcec227f2f457d2e P 32fd9da57e7f4036ab7fb5bf91543079 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:31:35.400741 18764 raft_consensus.cc:2454] T 4a2744552e5341c1bcec227f2f457d2e P b4feac633da947dda962b210a9a81e12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32fd9da57e7f4036ab7fb5bf91543079 in term 1.
I20260806 06:31:35.400780 19122 raft_consensus.cc:686] T 4a2744552e5341c1bcec227f2f457d2e P 32fd9da57e7f4036ab7fb5bf91543079 [term 1 LEADER]: Becoming Leader. State: Replica: 32fd9da57e7f4036ab7fb5bf91543079, State: Running, Role: LEADER
I20260806 06:31:35.400828 19122 consensus_queue.cc:237] T 4a2744552e5341c1bcec227f2f457d2e P 32fd9da57e7f4036ab7fb5bf91543079 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } } peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } }
I20260806 06:31:35.401403 18317 catalog_manager.cc:5495] T 4a2744552e5341c1bcec227f2f457d2e P 32fd9da57e7f4036ab7fb5bf91543079 reported cstate change: term changed from 0 to 1, leader changed from <none> to 32fd9da57e7f4036ab7fb5bf91543079 (127.7.43.65). New cstate: current_term: 1 leader_uuid: "32fd9da57e7f4036ab7fb5bf91543079" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } health_report { overall_health: UNKNOWN } } }
Aug 06 06:31:35 dist-test-slave-6zbq krb5kdc[18251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997894, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.43.126@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM
I20260806 06:31:35.430027 19144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:35.422958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40427 (local address 127.0.0.1:58236)
0806 06:31:35.423105 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:35.423116 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:35.423171 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:35.423293 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:35.423294 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:35.423557 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:35.423603 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:35.423605 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:35.424511 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:35.424512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:35.425035 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:35.425037 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:35.425481 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:35.425482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:35.425518 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:35.426026 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:35.426030 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:35.427537 (+ 1507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:35.429075 (+ 1538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:35.429081 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:35.429083 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:35.429276 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:35.429445 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:35.429445 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:35.429445 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:35.429494 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:35.429706 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:35.429709 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:35.429747 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 06:31:35.429792 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 06:31:35.429874 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":78,"threads_started":1}
I20260806 06:31:35.430085 18398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:35.423034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58236 (local address 127.7.43.126:40427)
0806 06:31:35.423086 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:35.423088 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:35.423180 (+ 92us) server_negotiation.cc:407] Connection header received
0806 06:31:35.423209 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:35.423210 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:35.423221 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:35.423290 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:35.423682 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:35.424434 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:35.425106 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:35.425422 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:35.425486 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:35.427620 (+ 2134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:35.427622 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:35.427622 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:35.427630 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:35.429005 (+ 1375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:35.429335 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:35.429337 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:35.429337 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:35.429393 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:35.429539 (+ 146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:35.429540 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:35.429541 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:35.429657 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:35.429717 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:35.429825 (+ 108us) server_negotiation.cc:299] Negotiation successful
0806 06:31:35.429936 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":30}
I20260806 06:31:35.431936 18317 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:58236:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 06:31:35.432159 18317 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:31:35.433535 18736 tablet_service.cc:1467] Processing CreateTablet for tablet fa400f953700465d8410b1f10a6db6e6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=86ccc2f206de4fae87c0ddc78d878696]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 06:31:35.433534 18521 tablet_service.cc:1467] Processing CreateTablet for tablet fa400f953700465d8410b1f10a6db6e6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=86ccc2f206de4fae87c0ddc78d878696]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 06:31:35.433678 18521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa400f953700465d8410b1f10a6db6e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:35.433679 18736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa400f953700465d8410b1f10a6db6e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:35.434839 19092 tablet_bootstrap.cc:492] T fa400f953700465d8410b1f10a6db6e6 P b4feac633da947dda962b210a9a81e12: Bootstrap starting.
I20260806 06:31:35.435148 18944 tablet_service.cc:1467] Processing CreateTablet for tablet fa400f953700465d8410b1f10a6db6e6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=86ccc2f206de4fae87c0ddc78d878696]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 06:31:35.435215 19092 tablet_bootstrap.cc:654] T fa400f953700465d8410b1f10a6db6e6 P b4feac633da947dda962b210a9a81e12: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:35.435272 18944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa400f953700465d8410b1f10a6db6e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:35.435742 19092 tablet_bootstrap.cc:492] T fa400f953700465d8410b1f10a6db6e6 P b4feac633da947dda962b210a9a81e12: No bootstrap required, opened a new log
I20260806 06:31:35.435786 19092 ts_tablet_manager.cc:1397] T fa400f953700465d8410b1f10a6db6e6 P b4feac633da947dda962b210a9a81e12: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:31:35.435916 19092 raft_consensus.cc:348] T fa400f953700465d8410b1f10a6db6e6 P b4feac633da947dda962b210a9a81e12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } } peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } }
I20260806 06:31:35.435972 19092 raft_consensus.cc:374] T fa400f953700465d8410b1f10a6db6e6 P b4feac633da947dda962b210a9a81e12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:35.435989 19092 raft_consensus.cc:729] T fa400f953700465d8410b1f10a6db6e6 P b4feac633da947dda962b210a9a81e12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4feac633da947dda962b210a9a81e12, State: Initialized, Role: FOLLOWER
I20260806 06:31:35.436031 19092 consensus_queue.cc:260] T fa400f953700465d8410b1f10a6db6e6 P b4feac633da947dda962b210a9a81e12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } } peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } }
I20260806 06:31:35.436120 19092 ts_tablet_manager.cc:1428] T fa400f953700465d8410b1f10a6db6e6 P b4feac633da947dda962b210a9a81e12: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:35.436245 19099 tablet_replica.cc:439] T fa400f953700465d8410b1f10a6db6e6 P b4feac633da947dda962b210a9a81e12: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } } peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } } }
I20260806 06:31:35.436302 19099 tablet_replica.cc:442] T fa400f953700465d8410b1f10a6db6e6 P b4feac633da947dda962b210a9a81e12: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:31:35.436563 19093 tablet_bootstrap.cc:492] T fa400f953700465d8410b1f10a6db6e6 P 32fd9da57e7f4036ab7fb5bf91543079: Bootstrap starting.
I20260806 06:31:35.436859 19093 tablet_bootstrap.cc:654] T fa400f953700465d8410b1f10a6db6e6 P 32fd9da57e7f4036ab7fb5bf91543079: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:35.437162 19097 tablet_bootstrap.cc:492] T fa400f953700465d8410b1f10a6db6e6 P 1f0f48ee9ccb47bd8d010429fe8902a0: Bootstrap starting.
I20260806 06:31:35.437536 19097 tablet_bootstrap.cc:654] T fa400f953700465d8410b1f10a6db6e6 P 1f0f48ee9ccb47bd8d010429fe8902a0: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:35.438100 19093 tablet_bootstrap.cc:492] T fa400f953700465d8410b1f10a6db6e6 P 32fd9da57e7f4036ab7fb5bf91543079: No bootstrap required, opened a new log
I20260806 06:31:35.438123 19097 tablet_bootstrap.cc:492] T fa400f953700465d8410b1f10a6db6e6 P 1f0f48ee9ccb47bd8d010429fe8902a0: No bootstrap required, opened a new log
I20260806 06:31:35.438158 19093 ts_tablet_manager.cc:1397] T fa400f953700465d8410b1f10a6db6e6 P 32fd9da57e7f4036ab7fb5bf91543079: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:31:35.438167 19097 ts_tablet_manager.cc:1397] T fa400f953700465d8410b1f10a6db6e6 P 1f0f48ee9ccb47bd8d010429fe8902a0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:31:35.438253 19093 raft_consensus.cc:348] T fa400f953700465d8410b1f10a6db6e6 P 32fd9da57e7f4036ab7fb5bf91543079 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } } peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } }
I20260806 06:31:35.438314 19093 raft_consensus.cc:374] T fa400f953700465d8410b1f10a6db6e6 P 32fd9da57e7f4036ab7fb5bf91543079 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:35.438333 19093 raft_consensus.cc:729] T fa400f953700465d8410b1f10a6db6e6 P 32fd9da57e7f4036ab7fb5bf91543079 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32fd9da57e7f4036ab7fb5bf91543079, State: Initialized, Role: FOLLOWER
I20260806 06:31:35.438333 19097 raft_consensus.cc:348] T fa400f953700465d8410b1f10a6db6e6 P 1f0f48ee9ccb47bd8d010429fe8902a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } } peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } }
I20260806 06:31:35.438386 19093 consensus_queue.cc:260] T fa400f953700465d8410b1f10a6db6e6 P 32fd9da57e7f4036ab7fb5bf91543079 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } } peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } }
I20260806 06:31:35.438428 19097 raft_consensus.cc:374] T fa400f953700465d8410b1f10a6db6e6 P 1f0f48ee9ccb47bd8d010429fe8902a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:35.438453 19097 raft_consensus.cc:729] T fa400f953700465d8410b1f10a6db6e6 P 1f0f48ee9ccb47bd8d010429fe8902a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f0f48ee9ccb47bd8d010429fe8902a0, State: Initialized, Role: FOLLOWER
I20260806 06:31:35.438460 19093 ts_tablet_manager.cc:1428] T fa400f953700465d8410b1f10a6db6e6 P 32fd9da57e7f4036ab7fb5bf91543079: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:35.438488 19097 consensus_queue.cc:260] T fa400f953700465d8410b1f10a6db6e6 P 1f0f48ee9ccb47bd8d010429fe8902a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } } peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } }
I20260806 06:31:35.438544 19122 tablet_replica.cc:439] T fa400f953700465d8410b1f10a6db6e6 P 32fd9da57e7f4036ab7fb5bf91543079: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } } peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } } }
I20260806 06:31:35.438576 19097 ts_tablet_manager.cc:1428] T fa400f953700465d8410b1f10a6db6e6 P 1f0f48ee9ccb47bd8d010429fe8902a0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:35.438601 19122 tablet_replica.cc:442] T fa400f953700465d8410b1f10a6db6e6 P 32fd9da57e7f4036ab7fb5bf91543079: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:31:35.438776 19102 tablet_replica.cc:439] T fa400f953700465d8410b1f10a6db6e6 P 1f0f48ee9ccb47bd8d010429fe8902a0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } } peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } } }
I20260806 06:31:35.438854 19102 tablet_replica.cc:442] T fa400f953700465d8410b1f10a6db6e6 P 1f0f48ee9ccb47bd8d010429fe8902a0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:31:35.471510 19099 raft_consensus.cc:482] T fa400f953700465d8410b1f10a6db6e6 P b4feac633da947dda962b210a9a81e12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:31:35.471582 19099 raft_consensus.cc:504] T fa400f953700465d8410b1f10a6db6e6 P b4feac633da947dda962b210a9a81e12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } } peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } }
I20260806 06:31:35.471844 19099 leader_election.cc:288] T fa400f953700465d8410b1f10a6db6e6 P b4feac633da947dda962b210a9a81e12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32fd9da57e7f4036ab7fb5bf91543079 (127.7.43.65:36049), 1f0f48ee9ccb47bd8d010429fe8902a0 (127.7.43.67:43555)
I20260806 06:31:35.474726 19091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:35.472076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:42895 (local address 127.7.43.65:36049)
0806 06:31:35.472128 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:35.472131 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:35.472225 (+ 94us) server_negotiation.cc:407] Connection header received
0806 06:31:35.472276 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:35.472277 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:35.472286 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:35.472331 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:31:35.472465 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:35.473033 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:35.474025 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:35.474414 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:35.474469 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:35.474480 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:35.474630 (+ 150us) server_negotiation.cc:299] Negotiation successful
0806 06:31:35.474665 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 06:31:35.474740 19087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:35.472088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:60049 (local address 127.7.43.67:43555)
0806 06:31:35.472155 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:35.472157 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:35.472220 (+ 63us) server_negotiation.cc:407] Connection header received
0806 06:31:35.472283 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:35.472284 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:35.472293 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:35.472340 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:31:35.472470 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:35.473016 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:35.474020 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:35.474429 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:35.474487 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:35.474496 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:35.474629 (+ 133us) server_negotiation.cc:299] Negotiation successful
0806 06:31:35.474676 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 06:31:35.474767 19153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:35.471997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.67:43555 (local address 127.7.43.66:60049)
0806 06:31:35.472155 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:35.472165 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:35.472223 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:35.472350 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:35.472351 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:35.472353 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:31:35.472418 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:35.472420 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:35.473101 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:35.473102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:35.473929 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:35.473931 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:35.474508 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:35.474509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:35.474569 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:35.474572 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:31:35.474618 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 06:31:35.474662 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":30,"thread_start_us":84,"threads_started":1}
I20260806 06:31:35.474936 19150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:35.471969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:36049 (local address 127.7.43.66:42895)
0806 06:31:35.472151 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:35.472162 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:35.472224 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:35.472339 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:35.472340 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:35.472343 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:31:35.472410 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:35.472413 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:35.473094 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:35.473095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:35.473935 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:35.473938 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:35.474479 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:35.474480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:35.474560 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:35.474565 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:31:35.474610 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 06:31:35.474662 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":41,"thread_start_us":97,"threads_started":1}
I20260806 06:31:35.474982 18978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa400f953700465d8410b1f10a6db6e6" candidate_uuid: "b4feac633da947dda962b210a9a81e12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" is_pre_election: true
I20260806 06:31:35.475068 18978 raft_consensus.cc:2454] T fa400f953700465d8410b1f10a6db6e6 P 1f0f48ee9ccb47bd8d010429fe8902a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4feac633da947dda962b210a9a81e12 in term 0.
I20260806 06:31:35.475196 18550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa400f953700465d8410b1f10a6db6e6" candidate_uuid: "b4feac633da947dda962b210a9a81e12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32fd9da57e7f4036ab7fb5bf91543079" is_pre_election: true
I20260806 06:31:35.475311 18550 raft_consensus.cc:2454] T fa400f953700465d8410b1f10a6db6e6 P 32fd9da57e7f4036ab7fb5bf91543079 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4feac633da947dda962b210a9a81e12 in term 0.
I20260806 06:31:35.475438 18663 leader_election.cc:302] T fa400f953700465d8410b1f10a6db6e6 P b4feac633da947dda962b210a9a81e12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f0f48ee9ccb47bd8d010429fe8902a0, b4feac633da947dda962b210a9a81e12; no voters:
I20260806 06:31:35.475541 19099 raft_consensus.cc:2781] T fa400f953700465d8410b1f10a6db6e6 P b4feac633da947dda962b210a9a81e12 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:31:35.475589 19099 raft_consensus.cc:482] T fa400f953700465d8410b1f10a6db6e6 P b4feac633da947dda962b210a9a81e12 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:31:35.475602 19099 raft_consensus.cc:3037] T fa400f953700465d8410b1f10a6db6e6 P b4feac633da947dda962b210a9a81e12 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:35.476266 19099 raft_consensus.cc:504] T fa400f953700465d8410b1f10a6db6e6 P b4feac633da947dda962b210a9a81e12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } } peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } }
I20260806 06:31:35.476383 19099 leader_election.cc:288] T fa400f953700465d8410b1f10a6db6e6 P b4feac633da947dda962b210a9a81e12 [CANDIDATE]: Term 1 election: Requested vote from peers 32fd9da57e7f4036ab7fb5bf91543079 (127.7.43.65:36049), 1f0f48ee9ccb47bd8d010429fe8902a0 (127.7.43.67:43555)
I20260806 06:31:35.476517 18978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa400f953700465d8410b1f10a6db6e6" candidate_uuid: "b4feac633da947dda962b210a9a81e12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0"
I20260806 06:31:35.476527 18550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa400f953700465d8410b1f10a6db6e6" candidate_uuid: "b4feac633da947dda962b210a9a81e12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32fd9da57e7f4036ab7fb5bf91543079"
I20260806 06:31:35.476585 18978 raft_consensus.cc:3037] T fa400f953700465d8410b1f10a6db6e6 P 1f0f48ee9ccb47bd8d010429fe8902a0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:35.476588 18550 raft_consensus.cc:3037] T fa400f953700465d8410b1f10a6db6e6 P 32fd9da57e7f4036ab7fb5bf91543079 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:35.477105 18978 raft_consensus.cc:2454] T fa400f953700465d8410b1f10a6db6e6 P 1f0f48ee9ccb47bd8d010429fe8902a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4feac633da947dda962b210a9a81e12 in term 1.
I20260806 06:31:35.477190 18550 raft_consensus.cc:2454] T fa400f953700465d8410b1f10a6db6e6 P 32fd9da57e7f4036ab7fb5bf91543079 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4feac633da947dda962b210a9a81e12 in term 1.
I20260806 06:31:35.477259 18663 leader_election.cc:302] T fa400f953700465d8410b1f10a6db6e6 P b4feac633da947dda962b210a9a81e12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f0f48ee9ccb47bd8d010429fe8902a0, b4feac633da947dda962b210a9a81e12; no voters:
I20260806 06:31:35.477344 19099 raft_consensus.cc:2781] T fa400f953700465d8410b1f10a6db6e6 P b4feac633da947dda962b210a9a81e12 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:31:35.477458 19099 raft_consensus.cc:686] T fa400f953700465d8410b1f10a6db6e6 P b4feac633da947dda962b210a9a81e12 [term 1 LEADER]: Becoming Leader. State: Replica: b4feac633da947dda962b210a9a81e12, State: Running, Role: LEADER
I20260806 06:31:35.477514 19099 consensus_queue.cc:237] T fa400f953700465d8410b1f10a6db6e6 P b4feac633da947dda962b210a9a81e12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } } peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } }
I20260806 06:31:35.477818 19154 tablet_replica.cc:439] T fa400f953700465d8410b1f10a6db6e6 P b4feac633da947dda962b210a9a81e12: TxnStatusTablet state changed. Reason: New leader b4feac633da947dda962b210a9a81e12. Latest consensus state: current_term: 1 leader_uuid: "b4feac633da947dda962b210a9a81e12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } } peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } } }
I20260806 06:31:35.477885 19154 tablet_replica.cc:442] T fa400f953700465d8410b1f10a6db6e6 P b4feac633da947dda962b210a9a81e12: This TxnStatusTablet replica's current role is: LEADER
I20260806 06:31:35.478024 19156 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 06:31:35.478122 19156 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 06:31:35.478134 18317 catalog_manager.cc:5495] T fa400f953700465d8410b1f10a6db6e6 P b4feac633da947dda962b210a9a81e12 reported cstate change: term changed from 0 to 1, leader changed from <none> to b4feac633da947dda962b210a9a81e12 (127.7.43.66). New cstate: current_term: 1 leader_uuid: "b4feac633da947dda962b210a9a81e12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } health_report { overall_health: UNKNOWN } } }
I20260806 06:31:35.486485 19089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:35.484330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56684 (local address 127.7.43.66:40585)
0806 06:31:35.484383 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:35.484385 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:35.484393 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:31:35.484439 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:35.484440 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:35.484455 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:35.484508 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:31:35.484604 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:35.485057 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:35.485763 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:35.485946 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:35.485997 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:35.486162 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:31:35.486250 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:31:35.486309 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:35.486402 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 06:31:35.486423 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 06:31:35.486497 19144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:35.484256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.66:40585 (local address 127.0.0.1:56684)
0806 06:31:35.484318 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:35.484329 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:35.484376 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:35.484511 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:35.484511 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:35.484512 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 06:31:35.484556 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:35.484558 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:35.485124 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:35.485125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:35.485682 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:35.485684 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:35.486003 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:35.486004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:35.486067 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:35.486083 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 06:31:35.486303 (+ 220us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 06:31:35.486312 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 06:31:35.486391 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 06:31:35.486433 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 06:31:35.487581 18978 raft_consensus.cc:1261] T fa400f953700465d8410b1f10a6db6e6 P 1f0f48ee9ccb47bd8d010429fe8902a0 [term 1 FOLLOWER]: Refusing update from remote peer b4feac633da947dda962b210a9a81e12: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:31:35.487691 18550 raft_consensus.cc:1261] T fa400f953700465d8410b1f10a6db6e6 P 32fd9da57e7f4036ab7fb5bf91543079 [term 1 FOLLOWER]: Refusing update from remote peer b4feac633da947dda962b210a9a81e12: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:31:35.487815 19154 consensus_queue.cc:1035] T fa400f953700465d8410b1f10a6db6e6 P b4feac633da947dda962b210a9a81e12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:31:35.487905 19099 consensus_queue.cc:1035] T fa400f953700465d8410b1f10a6db6e6 P b4feac633da947dda962b210a9a81e12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:31:35.488432 19102 tablet_replica.cc:439] T fa400f953700465d8410b1f10a6db6e6 P 1f0f48ee9ccb47bd8d010429fe8902a0: TxnStatusTablet state changed. Reason: New leader b4feac633da947dda962b210a9a81e12. Latest consensus state: current_term: 1 leader_uuid: "b4feac633da947dda962b210a9a81e12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } } peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } } }
I20260806 06:31:35.488507 19102 tablet_replica.cc:442] T fa400f953700465d8410b1f10a6db6e6 P 1f0f48ee9ccb47bd8d010429fe8902a0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:31:35.488698 19122 tablet_replica.cc:439] T fa400f953700465d8410b1f10a6db6e6 P 32fd9da57e7f4036ab7fb5bf91543079: TxnStatusTablet state changed. Reason: New leader b4feac633da947dda962b210a9a81e12. Latest consensus state: current_term: 1 leader_uuid: "b4feac633da947dda962b210a9a81e12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } } peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } } }
I20260806 06:31:35.488775 19122 tablet_replica.cc:442] T fa400f953700465d8410b1f10a6db6e6 P 32fd9da57e7f4036ab7fb5bf91543079: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:31:35.489375 19115 mvcc.cc:205] Tried to move back new op lower bound from 7315447379915509760 to 7315447379876384768. Current Snapshot: MvccSnapshot[applied={T|T < 7315447379915509760}]
I20260806 06:31:35.489466 19154 tablet_replica.cc:439] T fa400f953700465d8410b1f10a6db6e6 P b4feac633da947dda962b210a9a81e12: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b4feac633da947dda962b210a9a81e12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } } peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } } }
I20260806 06:31:35.489532 19154 tablet_replica.cc:442] T fa400f953700465d8410b1f10a6db6e6 P b4feac633da947dda962b210a9a81e12: This TxnStatusTablet replica's current role is: LEADER
I20260806 06:31:35.489576 19154 tablet_replica.cc:439] T fa400f953700465d8410b1f10a6db6e6 P b4feac633da947dda962b210a9a81e12: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b4feac633da947dda962b210a9a81e12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } } peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } } }
I20260806 06:31:35.489611 19154 tablet_replica.cc:442] T fa400f953700465d8410b1f10a6db6e6 P b4feac633da947dda962b210a9a81e12: This TxnStatusTablet replica's current role is: LEADER
I20260806 06:31:35.489660 19117 mvcc.cc:205] Tried to move back new op lower bound from 7315447379915509760 to 7315447379876384768. Current Snapshot: MvccSnapshot[applied={T|T < 7315447379915509760}]
I20260806 06:31:35.489758 19102 tablet_replica.cc:439] T fa400f953700465d8410b1f10a6db6e6 P 1f0f48ee9ccb47bd8d010429fe8902a0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b4feac633da947dda962b210a9a81e12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } } peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } } }
I20260806 06:31:35.489822 19102 tablet_replica.cc:442] T fa400f953700465d8410b1f10a6db6e6 P 1f0f48ee9ccb47bd8d010429fe8902a0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:31:35.490940 19114 mvcc.cc:205] Tried to move back new op lower bound from 7315447379915509760 to 7315447379876384768. Current Snapshot: MvccSnapshot[applied={T|T < 7315447379915509760}]
I20260806 06:31:35.491032 19122 tablet_replica.cc:439] T fa400f953700465d8410b1f10a6db6e6 P 32fd9da57e7f4036ab7fb5bf91543079: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b4feac633da947dda962b210a9a81e12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } } peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } } }
I20260806 06:31:35.491087 19122 tablet_replica.cc:442] T fa400f953700465d8410b1f10a6db6e6 P 32fd9da57e7f4036ab7fb5bf91543079: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:31:35.495167 19089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:35.492731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:51201 (local address 127.7.43.66:40585)
0806 06:31:35.492777 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:35.492779 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:35.492905 (+ 126us) server_negotiation.cc:407] Connection header received
0806 06:31:35.492961 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:35.492962 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:35.492978 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:35.493020 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:31:35.493130 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:35.493530 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:35.494644 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:35.494816 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:35.494860 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:35.495012 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:31:35.495033 (+ 21us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:31:35.495062 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:35.495113 (+ 51us) server_negotiation.cc:299] Negotiation successful
0806 06:31:35.495125 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 06:31:35.495379 19169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:35.492682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.66:40585 (local address 127.7.43.65:51201)
0806 06:31:35.492834 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:35.492847 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:35.492909 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:35.493034 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:35.493035 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:35.493036 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 06:31:35.493077 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:35.493078 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:35.493629 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:35.493630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:35.494555 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:35.494558 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:35.494875 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:35.494876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:35.494933 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:35.494948 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 06:31:35.495066 (+ 118us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 06:31:35.495067 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 06:31:35.495112 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 06:31:35.495156 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":88,"threads_started":1}
I20260806 06:31:35.497054 18978 raft_consensus.cc:1261] T 4a2744552e5341c1bcec227f2f457d2e P 1f0f48ee9ccb47bd8d010429fe8902a0 [term 1 FOLLOWER]: Refusing update from remote peer 32fd9da57e7f4036ab7fb5bf91543079: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:31:35.497123 18764 raft_consensus.cc:1261] T 4a2744552e5341c1bcec227f2f457d2e P b4feac633da947dda962b210a9a81e12 [term 1 FOLLOWER]: Refusing update from remote peer 32fd9da57e7f4036ab7fb5bf91543079: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:31:35.497231 19098 consensus_queue.cc:1035] T 4a2744552e5341c1bcec227f2f457d2e P 32fd9da57e7f4036ab7fb5bf91543079 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:31:35.497309 19122 consensus_queue.cc:1035] T 4a2744552e5341c1bcec227f2f457d2e P 32fd9da57e7f4036ab7fb5bf91543079 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:31:35.505457 19091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:35.503033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:38349 (local address 127.7.43.65:36049)
0806 06:31:35.503066 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:35.503068 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:35.503201 (+ 133us) server_negotiation.cc:407] Connection header received
0806 06:31:35.503252 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:35.503253 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:35.503263 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:35.503304 (+ 41us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:31:35.503431 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:35.503861 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:35.504885 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:35.505054 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:35.505105 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:35.505249 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:31:35.505282 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:31:35.505322 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:35.505379 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 06:31:35.505397 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 06:31:35.505498 19174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:35.502994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:36049 (local address 127.7.43.66:38349)
0806 06:31:35.503142 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:35.503150 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:35.503200 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:35.503304 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:35.503304 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:35.503306 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 06:31:35.503380 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:35.503382 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:35.503929 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:35.503930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:35.504760 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:35.504763 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:35.505112 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:35.505113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:35.505175 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:35.505189 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 06:31:35.505329 (+ 140us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 06:31:35.505330 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 06:31:35.505376 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 06:31:35.505414 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":75,"threads_started":1}
I20260806 06:31:35.513748 18317 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58234:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:31:35.513839 18317 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58234:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:31:35.514365 18317 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5c62144213b245f792a7cecb16ffa196: Sending DeleteTablet for 3 replicas of tablet 4a2744552e5341c1bcec227f2f457d2e
I20260806 06:31:35.514590 18521 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a2744552e5341c1bcec227f2f457d2e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:31:35 UTC) from {username='oryx', principal='oryx/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:46106
I20260806 06:31:35.514605 18944 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a2744552e5341c1bcec227f2f457d2e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:31:35 UTC) from {username='oryx', principal='oryx/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:50386
I20260806 06:31:35.514680 19126 tablet_replica.cc:330] T 4a2744552e5341c1bcec227f2f457d2e P 32fd9da57e7f4036ab7fb5bf91543079: stopping tablet replica
I20260806 06:31:35.514693 19127 tablet_replica.cc:330] T 4a2744552e5341c1bcec227f2f457d2e P 1f0f48ee9ccb47bd8d010429fe8902a0: stopping tablet replica
I20260806 06:31:35.514734 19127 raft_consensus.cc:2229] T 4a2744552e5341c1bcec227f2f457d2e P 1f0f48ee9ccb47bd8d010429fe8902a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:31:35.514729 19126 raft_consensus.cc:2229] T 4a2744552e5341c1bcec227f2f457d2e P 32fd9da57e7f4036ab7fb5bf91543079 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:31:35.514775 19127 raft_consensus.cc:2258] T 4a2744552e5341c1bcec227f2f457d2e P 1f0f48ee9ccb47bd8d010429fe8902a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:31:35.514786 19126 raft_consensus.cc:2258] T 4a2744552e5341c1bcec227f2f457d2e P 32fd9da57e7f4036ab7fb5bf91543079 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:31:35.515048 19126 ts_tablet_manager.cc:1905] T 4a2744552e5341c1bcec227f2f457d2e P 32fd9da57e7f4036ab7fb5bf91543079: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:31:35.515060 19127 ts_tablet_manager.cc:1905] T 4a2744552e5341c1bcec227f2f457d2e P 1f0f48ee9ccb47bd8d010429fe8902a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:31:35.516119 18736 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a2744552e5341c1bcec227f2f457d2e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:31:35 UTC) from {username='oryx', principal='oryx/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:56680
I20260806 06:31:35.516208 19128 tablet_replica.cc:330] T 4a2744552e5341c1bcec227f2f457d2e P b4feac633da947dda962b210a9a81e12: stopping tablet replica
I20260806 06:31:35.516260 19128 raft_consensus.cc:2229] T 4a2744552e5341c1bcec227f2f457d2e P b4feac633da947dda962b210a9a81e12 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:31:35.516294 19128 raft_consensus.cc:2258] T 4a2744552e5341c1bcec227f2f457d2e P b4feac633da947dda962b210a9a81e12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:31:35.516319 19126 ts_tablet_manager.cc:1918] T 4a2744552e5341c1bcec227f2f457d2e P 32fd9da57e7f4036ab7fb5bf91543079: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:31:35.516362 19126 log.cc:1196] T 4a2744552e5341c1bcec227f2f457d2e P 32fd9da57e7f4036ab7fb5bf91543079: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-0/wal/wals/4a2744552e5341c1bcec227f2f457d2e
I20260806 06:31:35.516476 19127 ts_tablet_manager.cc:1918] T 4a2744552e5341c1bcec227f2f457d2e P 1f0f48ee9ccb47bd8d010429fe8902a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:31:35.516541 19127 log.cc:1196] T 4a2744552e5341c1bcec227f2f457d2e P 1f0f48ee9ccb47bd8d010429fe8902a0: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-2/wal/wals/4a2744552e5341c1bcec227f2f457d2e
I20260806 06:31:35.516559 19126 ts_tablet_manager.cc:1939] T 4a2744552e5341c1bcec227f2f457d2e P 32fd9da57e7f4036ab7fb5bf91543079: Deleting consensus metadata
I20260806 06:31:35.516747 19127 ts_tablet_manager.cc:1939] T 4a2744552e5341c1bcec227f2f457d2e P 1f0f48ee9ccb47bd8d010429fe8902a0: Deleting consensus metadata
I20260806 06:31:35.516848 18300 catalog_manager.cc:4841] TS 32fd9da57e7f4036ab7fb5bf91543079 (127.7.43.65:36049): tablet 4a2744552e5341c1bcec227f2f457d2e (table test-table [id=818508665f4a480880538da05ec188ce]) successfully deleted
I20260806 06:31:35.516994 18296 catalog_manager.cc:4841] TS 1f0f48ee9ccb47bd8d010429fe8902a0 (127.7.43.67:43555): tablet 4a2744552e5341c1bcec227f2f457d2e (table test-table [id=818508665f4a480880538da05ec188ce]) successfully deleted
I20260806 06:31:35.517123 19128 ts_tablet_manager.cc:1905] T 4a2744552e5341c1bcec227f2f457d2e P b4feac633da947dda962b210a9a81e12: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:31:35.518263 19128 ts_tablet_manager.cc:1918] T 4a2744552e5341c1bcec227f2f457d2e P b4feac633da947dda962b210a9a81e12: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:31:35.518321 19128 log.cc:1196] T 4a2744552e5341c1bcec227f2f457d2e P b4feac633da947dda962b210a9a81e12: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-1/wal/wals/4a2744552e5341c1bcec227f2f457d2e
I20260806 06:31:35.518523 19128 ts_tablet_manager.cc:1939] T 4a2744552e5341c1bcec227f2f457d2e P b4feac633da947dda962b210a9a81e12: Deleting consensus metadata
I20260806 06:31:35.518793 18298 catalog_manager.cc:4841] TS b4feac633da947dda962b210a9a81e12 (127.7.43.66:40585): tablet 4a2744552e5341c1bcec227f2f457d2e (table test-table [id=818508665f4a480880538da05ec188ce]) successfully deleted
Aug 06 06:31:35 dist-test-slave-6zbq krb5kdc[18251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997895, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 06:31:35 dist-test-slave-6zbq krb5kdc[18251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:31:35 dist-test-slave-6zbq krb5kdc[18251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 06:31:35.538297 18398 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:35.533033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58240 (local address 127.7.43.126:40427)
0806 06:31:35.533077 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:35.533079 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:35.533225 (+ 146us) server_negotiation.cc:407] Connection header received
0806 06:31:35.533265 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:35.533265 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:35.533277 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:35.533336 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:35.533854 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:35.534697 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:35.535395 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:35.535679 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:35.535748 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:35.538238 (+ 2490us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58240: BlockingRecv error: recv got EOF from 127.0.0.1:58240 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 06 06:31:35 dist-test-slave-6zbq krb5kdc[18251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997895, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM
I20260806 06:31:35.545257 18398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:35.539025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58254 (local address 127.7.43.126:40427)
0806 06:31:35.539061 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:35.539063 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:35.539127 (+ 64us) server_negotiation.cc:407] Connection header received
0806 06:31:35.539193 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:35.539194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:35.539202 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:35.539260 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:35.539656 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:35.540093 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:35.540706 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:35.540885 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:35.540934 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:35.543115 (+ 2181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:35.543116 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:35.543117 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:35.543122 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:35.544354 (+ 1232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:35.544699 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:35.544699 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:35.544700 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:35.544746 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:35.544895 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:35.544895 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:35.544895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:35.544980 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:35.545040 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:35.545142 (+ 102us) server_negotiation.cc:299] Negotiation successful
0806 06:31:35.545192 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 06:31:35.546679 18317 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58254:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:31:35.546823 18317 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:31:35.547953 18521 tablet_service.cc:1467] Processing CreateTablet for tablet f72b9c30894c4752b436fb7de742bca1 (DEFAULT_TABLE table=test-table [id=53d84df942fa4ecdbdfed621751d7888]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:31:35.547991 18736 tablet_service.cc:1467] Processing CreateTablet for tablet f72b9c30894c4752b436fb7de742bca1 (DEFAULT_TABLE table=test-table [id=53d84df942fa4ecdbdfed621751d7888]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:31:35.548007 18944 tablet_service.cc:1467] Processing CreateTablet for tablet f72b9c30894c4752b436fb7de742bca1 (DEFAULT_TABLE table=test-table [id=53d84df942fa4ecdbdfed621751d7888]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:31:35.548074 18521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f72b9c30894c4752b436fb7de742bca1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:35.548095 18944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f72b9c30894c4752b436fb7de742bca1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:35.548099 18736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f72b9c30894c4752b436fb7de742bca1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:35.549263 19097 tablet_bootstrap.cc:492] T f72b9c30894c4752b436fb7de742bca1 P 1f0f48ee9ccb47bd8d010429fe8902a0: Bootstrap starting.
I20260806 06:31:35.549293 19092 tablet_bootstrap.cc:492] T f72b9c30894c4752b436fb7de742bca1 P b4feac633da947dda962b210a9a81e12: Bootstrap starting.
I20260806 06:31:35.549649 19097 tablet_bootstrap.cc:654] T f72b9c30894c4752b436fb7de742bca1 P 1f0f48ee9ccb47bd8d010429fe8902a0: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:35.549659 19092 tablet_bootstrap.cc:654] T f72b9c30894c4752b436fb7de742bca1 P b4feac633da947dda962b210a9a81e12: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:35.549949 19093 tablet_bootstrap.cc:492] T f72b9c30894c4752b436fb7de742bca1 P 32fd9da57e7f4036ab7fb5bf91543079: Bootstrap starting.
I20260806 06:31:35.550217 19093 tablet_bootstrap.cc:654] T f72b9c30894c4752b436fb7de742bca1 P 32fd9da57e7f4036ab7fb5bf91543079: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:35.550220 19097 tablet_bootstrap.cc:492] T f72b9c30894c4752b436fb7de742bca1 P 1f0f48ee9ccb47bd8d010429fe8902a0: No bootstrap required, opened a new log
I20260806 06:31:35.550271 19097 ts_tablet_manager.cc:1397] T f72b9c30894c4752b436fb7de742bca1 P 1f0f48ee9ccb47bd8d010429fe8902a0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:31:35.550472 19097 raft_consensus.cc:348] T f72b9c30894c4752b436fb7de742bca1 P 1f0f48ee9ccb47bd8d010429fe8902a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } } peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } }
I20260806 06:31:35.550545 19097 raft_consensus.cc:374] T f72b9c30894c4752b436fb7de742bca1 P 1f0f48ee9ccb47bd8d010429fe8902a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:35.550572 19097 raft_consensus.cc:729] T f72b9c30894c4752b436fb7de742bca1 P 1f0f48ee9ccb47bd8d010429fe8902a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f0f48ee9ccb47bd8d010429fe8902a0, State: Initialized, Role: FOLLOWER
I20260806 06:31:35.550619 19097 consensus_queue.cc:260] T f72b9c30894c4752b436fb7de742bca1 P 1f0f48ee9ccb47bd8d010429fe8902a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } } peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } }
I20260806 06:31:35.550712 19097 ts_tablet_manager.cc:1428] T f72b9c30894c4752b436fb7de742bca1 P 1f0f48ee9ccb47bd8d010429fe8902a0: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 06:31:35.550923 19093 tablet_bootstrap.cc:492] T f72b9c30894c4752b436fb7de742bca1 P 32fd9da57e7f4036ab7fb5bf91543079: No bootstrap required, opened a new log
I20260806 06:31:35.550972 19093 ts_tablet_manager.cc:1397] T f72b9c30894c4752b436fb7de742bca1 P 32fd9da57e7f4036ab7fb5bf91543079: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 06:31:35.551064 19093 raft_consensus.cc:348] T f72b9c30894c4752b436fb7de742bca1 P 32fd9da57e7f4036ab7fb5bf91543079 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } } peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } }
I20260806 06:31:35.551117 19093 raft_consensus.cc:374] T f72b9c30894c4752b436fb7de742bca1 P 32fd9da57e7f4036ab7fb5bf91543079 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:35.551143 19093 raft_consensus.cc:729] T f72b9c30894c4752b436fb7de742bca1 P 32fd9da57e7f4036ab7fb5bf91543079 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32fd9da57e7f4036ab7fb5bf91543079, State: Initialized, Role: FOLLOWER
I20260806 06:31:35.551180 19093 consensus_queue.cc:260] T f72b9c30894c4752b436fb7de742bca1 P 32fd9da57e7f4036ab7fb5bf91543079 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } } peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } }
I20260806 06:31:35.551263 19093 ts_tablet_manager.cc:1428] T f72b9c30894c4752b436fb7de742bca1 P 32fd9da57e7f4036ab7fb5bf91543079: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:35.551565 19092 tablet_bootstrap.cc:492] T f72b9c30894c4752b436fb7de742bca1 P b4feac633da947dda962b210a9a81e12: No bootstrap required, opened a new log
I20260806 06:31:35.551621 19092 ts_tablet_manager.cc:1397] T f72b9c30894c4752b436fb7de742bca1 P b4feac633da947dda962b210a9a81e12: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:31:35.551710 19092 raft_consensus.cc:348] T f72b9c30894c4752b436fb7de742bca1 P b4feac633da947dda962b210a9a81e12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } } peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } }
I20260806 06:31:35.551795 19092 raft_consensus.cc:374] T f72b9c30894c4752b436fb7de742bca1 P b4feac633da947dda962b210a9a81e12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:35.551812 19092 raft_consensus.cc:729] T f72b9c30894c4752b436fb7de742bca1 P b4feac633da947dda962b210a9a81e12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4feac633da947dda962b210a9a81e12, State: Initialized, Role: FOLLOWER
I20260806 06:31:35.551854 19092 consensus_queue.cc:260] T f72b9c30894c4752b436fb7de742bca1 P b4feac633da947dda962b210a9a81e12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } } peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } }
I20260806 06:31:35.551936 19092 ts_tablet_manager.cc:1428] T f72b9c30894c4752b436fb7de742bca1 P b4feac633da947dda962b210a9a81e12: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:35.603569 19122 raft_consensus.cc:482] T f72b9c30894c4752b436fb7de742bca1 P 32fd9da57e7f4036ab7fb5bf91543079 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:31:35.603667 19122 raft_consensus.cc:504] T f72b9c30894c4752b436fb7de742bca1 P 32fd9da57e7f4036ab7fb5bf91543079 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } } peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } }
I20260806 06:31:35.603822 19122 leader_election.cc:288] T f72b9c30894c4752b436fb7de742bca1 P 32fd9da57e7f4036ab7fb5bf91543079 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4feac633da947dda962b210a9a81e12 (127.7.43.66:40585), 1f0f48ee9ccb47bd8d010429fe8902a0 (127.7.43.67:43555)
I20260806 06:31:35.603977 18764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f72b9c30894c4752b436fb7de742bca1" candidate_uuid: "32fd9da57e7f4036ab7fb5bf91543079" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4feac633da947dda962b210a9a81e12" is_pre_election: true
I20260806 06:31:35.603986 18978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f72b9c30894c4752b436fb7de742bca1" candidate_uuid: "32fd9da57e7f4036ab7fb5bf91543079" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" is_pre_election: true
I20260806 06:31:35.604068 18764 raft_consensus.cc:2454] T f72b9c30894c4752b436fb7de742bca1 P b4feac633da947dda962b210a9a81e12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32fd9da57e7f4036ab7fb5bf91543079 in term 0.
I20260806 06:31:35.604086 18978 raft_consensus.cc:2454] T f72b9c30894c4752b436fb7de742bca1 P 1f0f48ee9ccb47bd8d010429fe8902a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32fd9da57e7f4036ab7fb5bf91543079 in term 0.
I20260806 06:31:35.604218 18448 leader_election.cc:302] T f72b9c30894c4752b436fb7de742bca1 P 32fd9da57e7f4036ab7fb5bf91543079 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f0f48ee9ccb47bd8d010429fe8902a0, 32fd9da57e7f4036ab7fb5bf91543079; no voters:
I20260806 06:31:35.604355 19122 raft_consensus.cc:2781] T f72b9c30894c4752b436fb7de742bca1 P 32fd9da57e7f4036ab7fb5bf91543079 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:31:35.604401 19122 raft_consensus.cc:482] T f72b9c30894c4752b436fb7de742bca1 P 32fd9da57e7f4036ab7fb5bf91543079 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:31:35.604421 19122 raft_consensus.cc:3037] T f72b9c30894c4752b436fb7de742bca1 P 32fd9da57e7f4036ab7fb5bf91543079 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:35.605125 19122 raft_consensus.cc:504] T f72b9c30894c4752b436fb7de742bca1 P 32fd9da57e7f4036ab7fb5bf91543079 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } } peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } }
I20260806 06:31:35.605247 19122 leader_election.cc:288] T f72b9c30894c4752b436fb7de742bca1 P 32fd9da57e7f4036ab7fb5bf91543079 [CANDIDATE]: Term 1 election: Requested vote from peers b4feac633da947dda962b210a9a81e12 (127.7.43.66:40585), 1f0f48ee9ccb47bd8d010429fe8902a0 (127.7.43.67:43555)
I20260806 06:31:35.605391 18764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f72b9c30894c4752b436fb7de742bca1" candidate_uuid: "32fd9da57e7f4036ab7fb5bf91543079" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4feac633da947dda962b210a9a81e12"
I20260806 06:31:35.605463 18764 raft_consensus.cc:3037] T f72b9c30894c4752b436fb7de742bca1 P b4feac633da947dda962b210a9a81e12 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:35.605520 18978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f72b9c30894c4752b436fb7de742bca1" candidate_uuid: "32fd9da57e7f4036ab7fb5bf91543079" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0"
I20260806 06:31:35.605595 18978 raft_consensus.cc:3037] T f72b9c30894c4752b436fb7de742bca1 P 1f0f48ee9ccb47bd8d010429fe8902a0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:35.606086 18764 raft_consensus.cc:2454] T f72b9c30894c4752b436fb7de742bca1 P b4feac633da947dda962b210a9a81e12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32fd9da57e7f4036ab7fb5bf91543079 in term 1.
I20260806 06:31:35.606258 18449 leader_election.cc:302] T f72b9c30894c4752b436fb7de742bca1 P 32fd9da57e7f4036ab7fb5bf91543079 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32fd9da57e7f4036ab7fb5bf91543079, b4feac633da947dda962b210a9a81e12; no voters:
I20260806 06:31:35.606287 18978 raft_consensus.cc:2454] T f72b9c30894c4752b436fb7de742bca1 P 1f0f48ee9ccb47bd8d010429fe8902a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32fd9da57e7f4036ab7fb5bf91543079 in term 1.
I20260806 06:31:35.606431 19122 raft_consensus.cc:2781] T f72b9c30894c4752b436fb7de742bca1 P 32fd9da57e7f4036ab7fb5bf91543079 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:31:35.606484 19122 raft_consensus.cc:686] T f72b9c30894c4752b436fb7de742bca1 P 32fd9da57e7f4036ab7fb5bf91543079 [term 1 LEADER]: Becoming Leader. State: Replica: 32fd9da57e7f4036ab7fb5bf91543079, State: Running, Role: LEADER
I20260806 06:31:35.606532 19122 consensus_queue.cc:237] T f72b9c30894c4752b436fb7de742bca1 P 32fd9da57e7f4036ab7fb5bf91543079 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } } peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } }
I20260806 06:31:35.607066 18317 catalog_manager.cc:5495] T f72b9c30894c4752b436fb7de742bca1 P 32fd9da57e7f4036ab7fb5bf91543079 reported cstate change: term changed from 0 to 1, leader changed from <none> to 32fd9da57e7f4036ab7fb5bf91543079 (127.7.43.65). New cstate: current_term: 1 leader_uuid: "32fd9da57e7f4036ab7fb5bf91543079" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } health_report { overall_health: UNKNOWN } } }
I20260806 06:31:35.624727 19091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:35.622741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46120 (local address 127.7.43.65:36049)
0806 06:31:35.622783 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:35.622785 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:35.622797 (+ 12us) server_negotiation.cc:407] Connection header received
0806 06:31:35.622849 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:35.622850 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:35.622861 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:35.622902 (+ 41us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:31:35.622997 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:35.623463 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:35.624189 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:35.624379 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:35.624430 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:35.624537 (+ 107us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:31:35.624563 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:31:35.624594 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:35.624661 (+ 67us) server_negotiation.cc:299] Negotiation successful
0806 06:31:35.624676 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:31:35.625348 18764 raft_consensus.cc:1261] T f72b9c30894c4752b436fb7de742bca1 P b4feac633da947dda962b210a9a81e12 [term 1 FOLLOWER]: Refusing update from remote peer 32fd9da57e7f4036ab7fb5bf91543079: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:31:35.625486 18978 raft_consensus.cc:1261] T f72b9c30894c4752b436fb7de742bca1 P 1f0f48ee9ccb47bd8d010429fe8902a0 [term 1 FOLLOWER]: Refusing update from remote peer 32fd9da57e7f4036ab7fb5bf91543079: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:31:35.625553 19109 consensus_queue.cc:1035] T f72b9c30894c4752b436fb7de742bca1 P 32fd9da57e7f4036ab7fb5bf91543079 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:31:35.625723 19109 consensus_queue.cc:1035] T f72b9c30894c4752b436fb7de742bca1 P 32fd9da57e7f4036ab7fb5bf91543079 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:31:35.627890 18317 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58254:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:31:35.627964 18317 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58254:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:31:35.628445 18317 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5c62144213b245f792a7cecb16ffa196: Sending DeleteTablet for 3 replicas of tablet f72b9c30894c4752b436fb7de742bca1
I20260806 06:31:35.628665 18521 tablet_service.cc:1514] Processing DeleteTablet for tablet f72b9c30894c4752b436fb7de742bca1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:31:35 UTC) from {username='oryx', principal='oryx/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:46106
I20260806 06:31:35.628677 18736 tablet_service.cc:1514] Processing DeleteTablet for tablet f72b9c30894c4752b436fb7de742bca1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:31:35 UTC) from {username='oryx', principal='oryx/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:56680
I20260806 06:31:35.628754 19128 tablet_replica.cc:330] T f72b9c30894c4752b436fb7de742bca1 P b4feac633da947dda962b210a9a81e12: stopping tablet replica
I20260806 06:31:35.628752 19126 tablet_replica.cc:330] T f72b9c30894c4752b436fb7de742bca1 P 32fd9da57e7f4036ab7fb5bf91543079: stopping tablet replica
I20260806 06:31:35.628759 18944 tablet_service.cc:1514] Processing DeleteTablet for tablet f72b9c30894c4752b436fb7de742bca1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:31:35 UTC) from {username='oryx', principal='oryx/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:50386
I20260806 06:31:35.628796 19128 raft_consensus.cc:2229] T f72b9c30894c4752b436fb7de742bca1 P b4feac633da947dda962b210a9a81e12 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:31:35.628813 19126 raft_consensus.cc:2229] T f72b9c30894c4752b436fb7de742bca1 P 32fd9da57e7f4036ab7fb5bf91543079 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:31:35.628830 19128 raft_consensus.cc:2258] T f72b9c30894c4752b436fb7de742bca1 P b4feac633da947dda962b210a9a81e12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:31:35.628868 19126 raft_consensus.cc:2258] T f72b9c30894c4752b436fb7de742bca1 P 32fd9da57e7f4036ab7fb5bf91543079 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:31:35.628894 19127 tablet_replica.cc:330] T f72b9c30894c4752b436fb7de742bca1 P 1f0f48ee9ccb47bd8d010429fe8902a0: stopping tablet replica
I20260806 06:31:35.628942 19127 raft_consensus.cc:2229] T f72b9c30894c4752b436fb7de742bca1 P 1f0f48ee9ccb47bd8d010429fe8902a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:31:35.628976 19127 raft_consensus.cc:2258] T f72b9c30894c4752b436fb7de742bca1 P 1f0f48ee9ccb47bd8d010429fe8902a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:31:35.629204 19127 ts_tablet_manager.cc:1905] T f72b9c30894c4752b436fb7de742bca1 P 1f0f48ee9ccb47bd8d010429fe8902a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:31:35.629204 19126 ts_tablet_manager.cc:1905] T f72b9c30894c4752b436fb7de742bca1 P 32fd9da57e7f4036ab7fb5bf91543079: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:31:35.629294 19128 ts_tablet_manager.cc:1905] T f72b9c30894c4752b436fb7de742bca1 P b4feac633da947dda962b210a9a81e12: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:31:35.630600 19127 ts_tablet_manager.cc:1918] T f72b9c30894c4752b436fb7de742bca1 P 1f0f48ee9ccb47bd8d010429fe8902a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:31:35.630666 19127 log.cc:1196] T f72b9c30894c4752b436fb7de742bca1 P 1f0f48ee9ccb47bd8d010429fe8902a0: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-2/wal/wals/f72b9c30894c4752b436fb7de742bca1
I20260806 06:31:35.630759 19128 ts_tablet_manager.cc:1918] T f72b9c30894c4752b436fb7de742bca1 P b4feac633da947dda962b210a9a81e12: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:31:35.630810 19128 log.cc:1196] T f72b9c30894c4752b436fb7de742bca1 P b4feac633da947dda962b210a9a81e12: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-1/wal/wals/f72b9c30894c4752b436fb7de742bca1
I20260806 06:31:35.630862 19127 ts_tablet_manager.cc:1939] T f72b9c30894c4752b436fb7de742bca1 P 1f0f48ee9ccb47bd8d010429fe8902a0: Deleting consensus metadata
I20260806 06:31:35.630897 19126 ts_tablet_manager.cc:1918] T f72b9c30894c4752b436fb7de742bca1 P 32fd9da57e7f4036ab7fb5bf91543079: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:31:35.630939 19126 log.cc:1196] T f72b9c30894c4752b436fb7de742bca1 P 32fd9da57e7f4036ab7fb5bf91543079: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-0/wal/wals/f72b9c30894c4752b436fb7de742bca1
I20260806 06:31:35.631026 19128 ts_tablet_manager.cc:1939] T f72b9c30894c4752b436fb7de742bca1 P b4feac633da947dda962b210a9a81e12: Deleting consensus metadata
I20260806 06:31:35.631167 19126 ts_tablet_manager.cc:1939] T f72b9c30894c4752b436fb7de742bca1 P 32fd9da57e7f4036ab7fb5bf91543079: Deleting consensus metadata
I20260806 06:31:35.631202 18296 catalog_manager.cc:4841] TS 1f0f48ee9ccb47bd8d010429fe8902a0 (127.7.43.67:43555): tablet f72b9c30894c4752b436fb7de742bca1 (table test-table [id=53d84df942fa4ecdbdfed621751d7888]) successfully deleted
I20260806 06:31:35.631307 18298 catalog_manager.cc:4841] TS b4feac633da947dda962b210a9a81e12 (127.7.43.66:40585): tablet f72b9c30894c4752b436fb7de742bca1 (table test-table [id=53d84df942fa4ecdbdfed621751d7888]) successfully deleted
I20260806 06:31:35.631448 18300 catalog_manager.cc:4841] TS 32fd9da57e7f4036ab7fb5bf91543079 (127.7.43.65:36049): tablet f72b9c30894c4752b436fb7de742bca1 (table test-table [id=53d84df942fa4ecdbdfed621751d7888]) successfully deleted
I20260806 06:31:35.632109 18317 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58254:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:31:35.632251 18317 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:31:35.633342 18944 tablet_service.cc:1467] Processing CreateTablet for tablet f7d790c9d46e47faaf0ca3d340434392 (DEFAULT_TABLE table=test-table [id=c595f9201e9f40f7a0200375ae5091dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:31:35.633338 18521 tablet_service.cc:1467] Processing CreateTablet for tablet f7d790c9d46e47faaf0ca3d340434392 (DEFAULT_TABLE table=test-table [id=c595f9201e9f40f7a0200375ae5091dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:31:35.633448 18521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7d790c9d46e47faaf0ca3d340434392. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:35.633447 18944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7d790c9d46e47faaf0ca3d340434392. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:35.634418 18736 tablet_service.cc:1467] Processing CreateTablet for tablet f7d790c9d46e47faaf0ca3d340434392 (DEFAULT_TABLE table=test-table [id=c595f9201e9f40f7a0200375ae5091dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:31:35.634521 18736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7d790c9d46e47faaf0ca3d340434392. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:35.634539 19097 tablet_bootstrap.cc:492] T f7d790c9d46e47faaf0ca3d340434392 P 1f0f48ee9ccb47bd8d010429fe8902a0: Bootstrap starting.
I20260806 06:31:35.635075 19097 tablet_bootstrap.cc:654] T f7d790c9d46e47faaf0ca3d340434392 P 1f0f48ee9ccb47bd8d010429fe8902a0: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:35.635442 19092 tablet_bootstrap.cc:492] T f7d790c9d46e47faaf0ca3d340434392 P b4feac633da947dda962b210a9a81e12: Bootstrap starting.
I20260806 06:31:35.635550 19093 tablet_bootstrap.cc:492] T f7d790c9d46e47faaf0ca3d340434392 P 32fd9da57e7f4036ab7fb5bf91543079: Bootstrap starting.
I20260806 06:31:35.635600 19097 tablet_bootstrap.cc:492] T f7d790c9d46e47faaf0ca3d340434392 P 1f0f48ee9ccb47bd8d010429fe8902a0: No bootstrap required, opened a new log
I20260806 06:31:35.635634 19097 ts_tablet_manager.cc:1397] T f7d790c9d46e47faaf0ca3d340434392 P 1f0f48ee9ccb47bd8d010429fe8902a0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:31:35.635705 19092 tablet_bootstrap.cc:654] T f7d790c9d46e47faaf0ca3d340434392 P b4feac633da947dda962b210a9a81e12: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:35.635742 19097 raft_consensus.cc:348] T f7d790c9d46e47faaf0ca3d340434392 P 1f0f48ee9ccb47bd8d010429fe8902a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } } peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } }
I20260806 06:31:35.635784 19097 raft_consensus.cc:374] T f7d790c9d46e47faaf0ca3d340434392 P 1f0f48ee9ccb47bd8d010429fe8902a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:35.635795 19097 raft_consensus.cc:729] T f7d790c9d46e47faaf0ca3d340434392 P 1f0f48ee9ccb47bd8d010429fe8902a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f0f48ee9ccb47bd8d010429fe8902a0, State: Initialized, Role: FOLLOWER
I20260806 06:31:35.635818 19093 tablet_bootstrap.cc:654] T f7d790c9d46e47faaf0ca3d340434392 P 32fd9da57e7f4036ab7fb5bf91543079: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:35.635828 19097 consensus_queue.cc:260] T f7d790c9d46e47faaf0ca3d340434392 P 1f0f48ee9ccb47bd8d010429fe8902a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } } peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } }
I20260806 06:31:35.635914 19097 ts_tablet_manager.cc:1428] T f7d790c9d46e47faaf0ca3d340434392 P 1f0f48ee9ccb47bd8d010429fe8902a0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:35.636292 19092 tablet_bootstrap.cc:492] T f7d790c9d46e47faaf0ca3d340434392 P b4feac633da947dda962b210a9a81e12: No bootstrap required, opened a new log
I20260806 06:31:35.636291 19093 tablet_bootstrap.cc:492] T f7d790c9d46e47faaf0ca3d340434392 P 32fd9da57e7f4036ab7fb5bf91543079: No bootstrap required, opened a new log
I20260806 06:31:35.636334 19092 ts_tablet_manager.cc:1397] T f7d790c9d46e47faaf0ca3d340434392 P b4feac633da947dda962b210a9a81e12: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260806 06:31:35.636337 19093 ts_tablet_manager.cc:1397] T f7d790c9d46e47faaf0ca3d340434392 P 32fd9da57e7f4036ab7fb5bf91543079: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 06:31:35.636425 19093 raft_consensus.cc:348] T f7d790c9d46e47faaf0ca3d340434392 P 32fd9da57e7f4036ab7fb5bf91543079 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } } peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } }
I20260806 06:31:35.636425 19092 raft_consensus.cc:348] T f7d790c9d46e47faaf0ca3d340434392 P b4feac633da947dda962b210a9a81e12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } } peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } }
I20260806 06:31:35.636485 19092 raft_consensus.cc:374] T f7d790c9d46e47faaf0ca3d340434392 P b4feac633da947dda962b210a9a81e12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:35.636487 19093 raft_consensus.cc:374] T f7d790c9d46e47faaf0ca3d340434392 P 32fd9da57e7f4036ab7fb5bf91543079 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:35.636513 19093 raft_consensus.cc:729] T f7d790c9d46e47faaf0ca3d340434392 P 32fd9da57e7f4036ab7fb5bf91543079 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32fd9da57e7f4036ab7fb5bf91543079, State: Initialized, Role: FOLLOWER
I20260806 06:31:35.636514 19092 raft_consensus.cc:729] T f7d790c9d46e47faaf0ca3d340434392 P b4feac633da947dda962b210a9a81e12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4feac633da947dda962b210a9a81e12, State: Initialized, Role: FOLLOWER
I20260806 06:31:35.636548 19093 consensus_queue.cc:260] T f7d790c9d46e47faaf0ca3d340434392 P 32fd9da57e7f4036ab7fb5bf91543079 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } } peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } }
I20260806 06:31:35.636554 19092 consensus_queue.cc:260] T f7d790c9d46e47faaf0ca3d340434392 P b4feac633da947dda962b210a9a81e12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } } peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } }
I20260806 06:31:35.636641 19093 ts_tablet_manager.cc:1428] T f7d790c9d46e47faaf0ca3d340434392 P 32fd9da57e7f4036ab7fb5bf91543079: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:35.636631 19092 ts_tablet_manager.cc:1428] T f7d790c9d46e47faaf0ca3d340434392 P b4feac633da947dda962b210a9a81e12: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:35.712272 19102 raft_consensus.cc:482] T f7d790c9d46e47faaf0ca3d340434392 P 1f0f48ee9ccb47bd8d010429fe8902a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:31:35.712363 19102 raft_consensus.cc:504] T f7d790c9d46e47faaf0ca3d340434392 P 1f0f48ee9ccb47bd8d010429fe8902a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } } peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } }
I20260806 06:31:35.712702 19102 leader_election.cc:288] T f7d790c9d46e47faaf0ca3d340434392 P 1f0f48ee9ccb47bd8d010429fe8902a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32fd9da57e7f4036ab7fb5bf91543079 (127.7.43.65:36049), b4feac633da947dda962b210a9a81e12 (127.7.43.66:40585)
I20260806 06:31:35.715512 19089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:35.712924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:56179 (local address 127.7.43.66:40585)
0806 06:31:35.712983 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:35.712985 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:35.713049 (+ 64us) server_negotiation.cc:407] Connection header received
0806 06:31:35.713097 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:35.713098 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:35.713107 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:35.713159 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:31:35.713326 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:35.713779 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:35.714846 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:35.715207 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:35.715269 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:35.715280 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:35.715417 (+ 137us) server_negotiation.cc:299] Negotiation successful
0806 06:31:35.715449 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 06:31:35.715520 19091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:35.712904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:42017 (local address 127.7.43.65:36049)
0806 06:31:35.712970 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:35.712972 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:35.713046 (+ 74us) server_negotiation.cc:407] Connection header received
0806 06:31:35.713110 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:35.713111 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:35.713119 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:35.713179 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:31:35.713332 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:35.713788 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:35.714841 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:35.715197 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:35.715256 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:35.715268 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:35.715418 (+ 150us) server_negotiation.cc:299] Negotiation successful
0806 06:31:35.715456 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 06:31:35.715535 19207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:35.712840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.66:40585 (local address 127.7.43.67:56179)
0806 06:31:35.712974 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:35.712983 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:35.713043 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:35.713172 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:35.713173 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:35.713175 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:31:35.713265 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:35.713267 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:35.713868 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:35.713869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:35.714759 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:35.714761 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:35.715300 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:35.715301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:35.715352 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:35.715356 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:31:35.715400 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 06:31:35.715440 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":52,"thread_start_us":67,"threads_started":1}
I20260806 06:31:35.715538 19206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:35.712840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:36049 (local address 127.7.43.67:42017)
0806 06:31:35.712971 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:35.712983 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:35.713051 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:35.713183 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:35.713184 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:35.713186 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:31:35.713274 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:35.713276 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:35.713879 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:35.713880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:35.714762 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:35.714765 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:35.715294 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:35.715295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:35.715356 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:35.715359 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:31:35.715406 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 06:31:35.715446 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"mutex_wait_us":21,"spinlock_wait_cycles":1792,"thread_start_us":66,"threads_started":1}
I20260806 06:31:35.715785 18764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7d790c9d46e47faaf0ca3d340434392" candidate_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4feac633da947dda962b210a9a81e12" is_pre_election: true
I20260806 06:31:35.715883 18764 raft_consensus.cc:2454] T f7d790c9d46e47faaf0ca3d340434392 P b4feac633da947dda962b210a9a81e12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f0f48ee9ccb47bd8d010429fe8902a0 in term 0.
I20260806 06:31:35.715962 18550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7d790c9d46e47faaf0ca3d340434392" candidate_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32fd9da57e7f4036ab7fb5bf91543079" is_pre_election: true
I20260806 06:31:35.716061 18550 raft_consensus.cc:2454] T f7d790c9d46e47faaf0ca3d340434392 P 32fd9da57e7f4036ab7fb5bf91543079 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f0f48ee9ccb47bd8d010429fe8902a0 in term 0.
I20260806 06:31:35.716063 18875 leader_election.cc:302] T f7d790c9d46e47faaf0ca3d340434392 P 1f0f48ee9ccb47bd8d010429fe8902a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f0f48ee9ccb47bd8d010429fe8902a0, b4feac633da947dda962b210a9a81e12; no voters:
I20260806 06:31:35.716202 19102 raft_consensus.cc:2781] T f7d790c9d46e47faaf0ca3d340434392 P 1f0f48ee9ccb47bd8d010429fe8902a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:31:35.716245 19102 raft_consensus.cc:482] T f7d790c9d46e47faaf0ca3d340434392 P 1f0f48ee9ccb47bd8d010429fe8902a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:31:35.716270 19102 raft_consensus.cc:3037] T f7d790c9d46e47faaf0ca3d340434392 P 1f0f48ee9ccb47bd8d010429fe8902a0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:35.716902 19102 raft_consensus.cc:504] T f7d790c9d46e47faaf0ca3d340434392 P 1f0f48ee9ccb47bd8d010429fe8902a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } } peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } }
I20260806 06:31:35.717013 19102 leader_election.cc:288] T f7d790c9d46e47faaf0ca3d340434392 P 1f0f48ee9ccb47bd8d010429fe8902a0 [CANDIDATE]: Term 1 election: Requested vote from peers 32fd9da57e7f4036ab7fb5bf91543079 (127.7.43.65:36049), b4feac633da947dda962b210a9a81e12 (127.7.43.66:40585)
I20260806 06:31:35.717160 18764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7d790c9d46e47faaf0ca3d340434392" candidate_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4feac633da947dda962b210a9a81e12"
I20260806 06:31:35.717173 18550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7d790c9d46e47faaf0ca3d340434392" candidate_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32fd9da57e7f4036ab7fb5bf91543079"
I20260806 06:31:35.717227 18764 raft_consensus.cc:3037] T f7d790c9d46e47faaf0ca3d340434392 P b4feac633da947dda962b210a9a81e12 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:35.717236 18550 raft_consensus.cc:3037] T f7d790c9d46e47faaf0ca3d340434392 P 32fd9da57e7f4036ab7fb5bf91543079 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:35.717772 18764 raft_consensus.cc:2454] T f7d790c9d46e47faaf0ca3d340434392 P b4feac633da947dda962b210a9a81e12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f0f48ee9ccb47bd8d010429fe8902a0 in term 1.
I20260806 06:31:35.717823 18550 raft_consensus.cc:2454] T f7d790c9d46e47faaf0ca3d340434392 P 32fd9da57e7f4036ab7fb5bf91543079 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f0f48ee9ccb47bd8d010429fe8902a0 in term 1.
I20260806 06:31:35.717980 18877 leader_election.cc:302] T f7d790c9d46e47faaf0ca3d340434392 P 1f0f48ee9ccb47bd8d010429fe8902a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f0f48ee9ccb47bd8d010429fe8902a0, 32fd9da57e7f4036ab7fb5bf91543079; no voters:
I20260806 06:31:35.718075 19102 raft_consensus.cc:2781] T f7d790c9d46e47faaf0ca3d340434392 P 1f0f48ee9ccb47bd8d010429fe8902a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:31:35.718199 19102 raft_consensus.cc:686] T f7d790c9d46e47faaf0ca3d340434392 P 1f0f48ee9ccb47bd8d010429fe8902a0 [term 1 LEADER]: Becoming Leader. State: Replica: 1f0f48ee9ccb47bd8d010429fe8902a0, State: Running, Role: LEADER
I20260806 06:31:35.718266 19102 consensus_queue.cc:237] T f7d790c9d46e47faaf0ca3d340434392 P 1f0f48ee9ccb47bd8d010429fe8902a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } } peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } }
I20260806 06:31:35.718958 18317 catalog_manager.cc:5495] T f7d790c9d46e47faaf0ca3d340434392 P 1f0f48ee9ccb47bd8d010429fe8902a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f0f48ee9ccb47bd8d010429fe8902a0 (127.7.43.67). New cstate: current_term: 1 leader_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f0f48ee9ccb47bd8d010429fe8902a0" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 43555 } health_report { overall_health: HEALTHY } } }
I20260806 06:31:35.728683 19087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:35.726820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50388 (local address 127.7.43.67:43555)
0806 06:31:35.726864 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:35.726866 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:35.726876 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:31:35.726942 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:35.726943 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:35.726958 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:35.727012 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:31:35.727090 (+ 78us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:35.727500 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:35.728146 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:35.728322 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:35.728371 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:35.728456 (+ 85us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:31:35.728544 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:31:35.728592 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:35.728618 (+ 26us) server_negotiation.cc:299] Negotiation successful
0806 06:31:35.728639 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 06:31:35.732803 19089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:35.730828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:37721 (local address 127.7.43.66:40585)
0806 06:31:35.730875 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:35.730876 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:35.730941 (+ 65us) server_negotiation.cc:407] Connection header received
0806 06:31:35.730986 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:35.730986 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:35.730993 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:35.731035 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:31:35.731110 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:35.731571 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:35.732280 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:35.732490 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:35.732538 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:35.732657 (+ 119us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:31:35.732680 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:31:35.732710 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:35.732747 (+ 37us) server_negotiation.cc:299] Negotiation successful
0806 06:31:35.732759 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 06:31:35.732833 19215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:35.730766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.66:40585 (local address 127.7.43.67:37721)
0806 06:31:35.730892 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:35.730901 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:35.730951 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:35.731036 (+ 85us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:35.731037 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:35.731038 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 06:31:35.731077 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:35.731078 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:35.731628 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:35.731629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:35.732227 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:35.732229 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:35.732543 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:35.732543 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:35.732585 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:35.732598 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 06:31:35.732716 (+ 118us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 06:31:35.732717 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 06:31:35.732744 (+ 27us) client_negotiation.cc:240] Negotiation successful
0806 06:31:35.732771 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":65,"threads_started":1}
I20260806 06:31:35.734766 18764 raft_consensus.cc:1261] T f7d790c9d46e47faaf0ca3d340434392 P b4feac633da947dda962b210a9a81e12 [term 1 FOLLOWER]: Refusing update from remote peer 1f0f48ee9ccb47bd8d010429fe8902a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:31:35.734766 18550 raft_consensus.cc:1261] T f7d790c9d46e47faaf0ca3d340434392 P 32fd9da57e7f4036ab7fb5bf91543079 [term 1 FOLLOWER]: Refusing update from remote peer 1f0f48ee9ccb47bd8d010429fe8902a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:31:35.735045 19102 consensus_queue.cc:1035] T f7d790c9d46e47faaf0ca3d340434392 P 1f0f48ee9ccb47bd8d010429fe8902a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32fd9da57e7f4036ab7fb5bf91543079" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:31:35.735172 19209 consensus_queue.cc:1035] T f7d790c9d46e47faaf0ca3d340434392 P 1f0f48ee9ccb47bd8d010429fe8902a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4feac633da947dda962b210a9a81e12" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:31:35.741039 19087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:35.739204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:39687 (local address 127.7.43.67:43555)
0806 06:31:35.739244 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:35.739246 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:35.739254 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:31:35.739294 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:35.739295 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:35.739302 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:35.739342 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:31:35.739458 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:35.739941 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:35.740573 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:35.740741 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:35.740780 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:35.740867 (+ 87us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:31:35.740886 (+ 19us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:31:35.740916 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:35.740961 (+ 45us) server_negotiation.cc:299] Negotiation successful
0806 06:31:35.740977 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:31:35.741055 19174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:35.739153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.67:43555 (local address 127.7.43.66:39687)
0806 06:31:35.739197 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:35.739207 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:35.739247 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:35.739360 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:35.739361 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:35.739361 (+ 0us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 06:31:35.739396 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:35.739397 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:35.739999 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:35.740000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:35.740516 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:35.740517 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:35.740786 (+ 269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:35.740787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:35.740820 (+ 33us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:35.740825 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 06:31:35.740929 (+ 104us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 06:31:35.740930 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 06:31:35.740958 (+ 28us) client_negotiation.cc:240] Negotiation successful
0806 06:31:35.740990 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260806 06:31:35.747991 18319 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58254:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:31:35.748086 18319 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58254:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:31:35.748621 18319 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5c62144213b245f792a7cecb16ffa196: Sending DeleteTablet for 3 replicas of tablet f7d790c9d46e47faaf0ca3d340434392
I20260806 06:31:35.748888 18736 tablet_service.cc:1514] Processing DeleteTablet for tablet f7d790c9d46e47faaf0ca3d340434392 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:31:35 UTC) from {username='oryx', principal='oryx/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:56680
I20260806 06:31:35.748888 18944 tablet_service.cc:1514] Processing DeleteTablet for tablet f7d790c9d46e47faaf0ca3d340434392 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:31:35 UTC) from {username='oryx', principal='oryx/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:50386
I20260806 06:31:35.748906 18521 tablet_service.cc:1514] Processing DeleteTablet for tablet f7d790c9d46e47faaf0ca3d340434392 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:31:35 UTC) from {username='oryx', principal='oryx/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:46106
I20260806 06:31:35.748967 19128 tablet_replica.cc:330] T f7d790c9d46e47faaf0ca3d340434392 P b4feac633da947dda962b210a9a81e12: stopping tablet replica
I20260806 06:31:35.748998 19127 tablet_replica.cc:330] T f7d790c9d46e47faaf0ca3d340434392 P 1f0f48ee9ccb47bd8d010429fe8902a0: stopping tablet replica
I20260806 06:31:35.749013 19128 raft_consensus.cc:2229] T f7d790c9d46e47faaf0ca3d340434392 P b4feac633da947dda962b210a9a81e12 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:31:35.749040 19128 raft_consensus.cc:2258] T f7d790c9d46e47faaf0ca3d340434392 P b4feac633da947dda962b210a9a81e12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:31:35.749049 19127 raft_consensus.cc:2229] T f7d790c9d46e47faaf0ca3d340434392 P 1f0f48ee9ccb47bd8d010429fe8902a0 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:31:35.749123 19127 raft_consensus.cc:2258] T f7d790c9d46e47faaf0ca3d340434392 P 1f0f48ee9ccb47bd8d010429fe8902a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:31:35.749204 19126 tablet_replica.cc:330] T f7d790c9d46e47faaf0ca3d340434392 P 32fd9da57e7f4036ab7fb5bf91543079: stopping tablet replica
I20260806 06:31:35.749264 19126 raft_consensus.cc:2229] T f7d790c9d46e47faaf0ca3d340434392 P 32fd9da57e7f4036ab7fb5bf91543079 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:31:35.749300 19126 raft_consensus.cc:2258] T f7d790c9d46e47faaf0ca3d340434392 P 32fd9da57e7f4036ab7fb5bf91543079 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:31:35.749367 19128 ts_tablet_manager.cc:1905] T f7d790c9d46e47faaf0ca3d340434392 P b4feac633da947dda962b210a9a81e12: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:31:35.749639 19126 ts_tablet_manager.cc:1905] T f7d790c9d46e47faaf0ca3d340434392 P 32fd9da57e7f4036ab7fb5bf91543079: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:31:35.750723 19128 ts_tablet_manager.cc:1918] T f7d790c9d46e47faaf0ca3d340434392 P b4feac633da947dda962b210a9a81e12: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:31:35.750773 19128 log.cc:1196] T f7d790c9d46e47faaf0ca3d340434392 P b4feac633da947dda962b210a9a81e12: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-1/wal/wals/f7d790c9d46e47faaf0ca3d340434392
I20260806 06:31:35.750954 19128 ts_tablet_manager.cc:1939] T f7d790c9d46e47faaf0ca3d340434392 P b4feac633da947dda962b210a9a81e12: Deleting consensus metadata
I20260806 06:31:35.751129 19126 ts_tablet_manager.cc:1918] T f7d790c9d46e47faaf0ca3d340434392 P 32fd9da57e7f4036ab7fb5bf91543079: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:31:35.751189 19126 log.cc:1196] T f7d790c9d46e47faaf0ca3d340434392 P 32fd9da57e7f4036ab7fb5bf91543079: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-0/wal/wals/f7d790c9d46e47faaf0ca3d340434392
I20260806 06:31:35.751257 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 18423
I20260806 06:31:35.751385 19126 ts_tablet_manager.cc:1939] T f7d790c9d46e47faaf0ca3d340434392 P 32fd9da57e7f4036ab7fb5bf91543079: Deleting consensus metadata
I20260806 06:31:35.760298 19127 ts_tablet_manager.cc:1905] T f7d790c9d46e47faaf0ca3d340434392 P 1f0f48ee9ccb47bd8d010429fe8902a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:31:35.761251 18298 catalog_manager.cc:4841] TS b4feac633da947dda962b210a9a81e12 (127.7.43.66:40585): tablet f7d790c9d46e47faaf0ca3d340434392 (table test-table [id=c595f9201e9f40f7a0200375ae5091dc]) successfully deleted
I20260806 06:31:35.761749 19127 ts_tablet_manager.cc:1918] T f7d790c9d46e47faaf0ca3d340434392 P 1f0f48ee9ccb47bd8d010429fe8902a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:31:35.761814 19127 log.cc:1196] T f7d790c9d46e47faaf0ca3d340434392 P 1f0f48ee9ccb47bd8d010429fe8902a0: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785997861828271-7341-0/minicluster-data/ts-2/wal/wals/f7d790c9d46e47faaf0ca3d340434392
I20260806 06:31:35.762002 19127 ts_tablet_manager.cc:1939] T f7d790c9d46e47faaf0ca3d340434392 P 1f0f48ee9ccb47bd8d010429fe8902a0: Deleting consensus metadata
I20260806 06:31:35.762388 18296 catalog_manager.cc:4841] TS 1f0f48ee9ccb47bd8d010429fe8902a0 (127.7.43.67:43555): tablet f7d790c9d46e47faaf0ca3d340434392 (table test-table [id=c595f9201e9f40f7a0200375ae5091dc]) successfully deleted
W20260806 06:31:35.767096 18300 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.43.65:36049 (error 108)
I20260806 06:31:35.767302 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 18640
I20260806 06:31:35.769766 19225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:35.769551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:36049 (local address 127.0.0.1:60478)
0806 06:31:35.769668 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:35.769721 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.65:36049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":48,"threads_started":1}
W20260806 06:31:35.769975 18300 catalog_manager.cc:4584] TS 32fd9da57e7f4036ab7fb5bf91543079 (127.7.43.65:36049): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f7d790c9d46e47faaf0ca3d340434392: Network error: Client connection negotiation failed: client connection to 127.7.43.65:36049: connect: Connection refused (error 111)
I20260806 06:31:35.777920 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 18851
I20260806 06:31:35.789801 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 18272
2026-08-06T06:31:35Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2666 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785997861828271-7341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:31:35 dist-test-slave-6zbq krb5kdc[19228](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:31:35 dist-test-slave-6zbq krb5kdc[19228](info): set up 2 sockets
Aug 06 06:31:35 dist-test-slave-6zbq krb5kdc[19228](info): commencing operation
krb5kdc: starting...
W20260806 06:31:36.904902 7341 mini_kdc.cc:121] Time spent starting KDC: real 1.088s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Aug 06 06:31:36 dist-test-slave-6zbq krb5kdc[19228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997896, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:31:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:31:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.43.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
I20260806 06:31:37.009737 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:42553
--webserver_interface=127.7.43.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:41695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.43.126:42553
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:31:37.088364 19251 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:37.088541 19251 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:37.088572 19251 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:37.089668 19251 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:31:37.089706 19251 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:37.089726 19251 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:37.089741 19251 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:31:37.089757 19251 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:31:37.091265 19251 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:41695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785997861828271-7341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.43.126:42553
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:42553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.43.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19251
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:37.091492 19251 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:37.091670 19251 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:37.095921 19262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:37.095930 19261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:37.096033 19251 server_base.cc:1034] running on GCE node
W20260806 06:31:37.095937 19264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:37.096390 19251 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:37.096643 19251 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:37.097781 19251 hybrid_clock.cc:648] HybridClock initialized: now 1785997897097765 us; error 30 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-06T06:31:37Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1309 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:31:37 dist-test-slave-6zbq krb5kdc[19273](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:31:37 dist-test-slave-6zbq krb5kdc[19273](info): set up 2 sockets
Aug 06 06:31:37 dist-test-slave-6zbq krb5kdc[19273](info): commencing operation
krb5kdc: starting...
W20260806 06:31:38.213044 7341 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Aug 06 06:31:38 dist-test-slave-6zbq krb5kdc[19273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997898, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:31:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:31:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.43.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
I20260806 06:31:38.317541 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:38047
--webserver_interface=127.7.43.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:37075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.43.126:38047
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:31:38.394627 19290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:38.394800 19290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:38.394820 19290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:38.395896 19290 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:31:38.395936 19290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:38.395948 19290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:38.395959 19290 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:31:38.395982 19290 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:31:38.397374 19290 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:37075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.43.126:38047
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:38047
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.43.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19290
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:38.397579 19290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:38.397773 19290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:38.402105 19296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:38.402206 19297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:38.402156 19299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:38.402266 19290 server_base.cc:1034] running on GCE node
I20260806 06:31:38.402486 19290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:38.402736 19290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:38.403856 19290 hybrid_clock.cc:648] HybridClock initialized: now 1785997898403847 us; error 27 us; skew 500 ppm
Aug 06 06:31:38 dist-test-slave-6zbq krb5kdc[19273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997898, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:38.406029 19290 init.cc:377] Logged in from keytab as kudu/127.7.43.126@KRBTEST.COM (short username kudu)
I20260806 06:31:38.406474 19290 webserver.cc:460] Webserver started at http://127.7.43.126:43139/ using document root <none> and password file <none>
I20260806 06:31:38.406670 19290 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:38.406716 19290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:38.406836 19290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:38.407768 19290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/master-0/data/instance:
uuid: "dadea2b0ec8d490eb68a73fe407f3e90"
format_stamp: "Formatted at 2026-08-06 06:31:38 on dist-test-slave-6zbq"
server_key: "ce7f8de951ec3f0141299614ca1efb88"
server_key_iv: "041c5b1dad3d90507d6ad3e4855e94fb"
server_key_version: "encryptionkey@0"
I20260806 06:31:38.408079 19290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/master-0/wal/instance:
uuid: "dadea2b0ec8d490eb68a73fe407f3e90"
format_stamp: "Formatted at 2026-08-06 06:31:38 on dist-test-slave-6zbq"
server_key: "ce7f8de951ec3f0141299614ca1efb88"
server_key_iv: "041c5b1dad3d90507d6ad3e4855e94fb"
server_key_version: "encryptionkey@0"
I20260806 06:31:38.409228 19290 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:31:38.409823 19308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:38.409972 19290 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:31:38.410051 19290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "dadea2b0ec8d490eb68a73fe407f3e90"
format_stamp: "Formatted at 2026-08-06 06:31:38 on dist-test-slave-6zbq"
server_key: "ce7f8de951ec3f0141299614ca1efb88"
server_key_iv: "041c5b1dad3d90507d6ad3e4855e94fb"
server_key_version: "encryptionkey@0"
I20260806 06:31:38.410123 19290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:38.430730 19290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:38.432605 19290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:38.432731 19290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:38.436324 19290 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.126:38047
I20260806 06:31:38.436360 19388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.126:38047 every 8 connection(s)
I20260806 06:31:38.436637 19290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:31:38.437397 19389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:38.439486 19389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dadea2b0ec8d490eb68a73fe407f3e90: Bootstrap starting.
I20260806 06:31:38.440002 19389 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dadea2b0ec8d490eb68a73fe407f3e90: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:38.440271 19389 log.cc:824] T 00000000000000000000000000000000 P dadea2b0ec8d490eb68a73fe407f3e90: Log is configured to *not* fsync() on all Append() calls
I20260806 06:31:38.440857 19389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dadea2b0ec8d490eb68a73fe407f3e90: No bootstrap required, opened a new log
I20260806 06:31:38.441450 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 19290
I20260806 06:31:38.441556 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/master-0/wal/instance
I20260806 06:31:38.441735 7341 external_mini_cluster.cc:1428] Setting key e455a7c37bc6152b6b03bc3ee034d1a2
I20260806 06:31:38.442821 19389 raft_consensus.cc:348] T 00000000000000000000000000000000 P dadea2b0ec8d490eb68a73fe407f3e90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dadea2b0ec8d490eb68a73fe407f3e90" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 38047 } }
I20260806 06:31:38.442981 19389 raft_consensus.cc:374] T 00000000000000000000000000000000 P dadea2b0ec8d490eb68a73fe407f3e90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:38.443017 19389 raft_consensus.cc:729] T 00000000000000000000000000000000 P dadea2b0ec8d490eb68a73fe407f3e90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dadea2b0ec8d490eb68a73fe407f3e90, State: Initialized, Role: FOLLOWER
I20260806 06:31:38.443173 19389 consensus_queue.cc:260] T 00000000000000000000000000000000 P dadea2b0ec8d490eb68a73fe407f3e90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dadea2b0ec8d490eb68a73fe407f3e90" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 38047 } }
I20260806 06:31:38.443240 19389 raft_consensus.cc:388] T 00000000000000000000000000000000 P dadea2b0ec8d490eb68a73fe407f3e90 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:31:38.443261 19389 raft_consensus.cc:482] T 00000000000000000000000000000000 P dadea2b0ec8d490eb68a73fe407f3e90 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:31:38.443291 19389 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dadea2b0ec8d490eb68a73fe407f3e90 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:38.444252 19389 raft_consensus.cc:504] T 00000000000000000000000000000000 P dadea2b0ec8d490eb68a73fe407f3e90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dadea2b0ec8d490eb68a73fe407f3e90" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 38047 } }
I20260806 06:31:38.444357 19389 leader_election.cc:302] T 00000000000000000000000000000000 P dadea2b0ec8d490eb68a73fe407f3e90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dadea2b0ec8d490eb68a73fe407f3e90; no voters:
I20260806 06:31:38.444504 19389 leader_election.cc:288] T 00000000000000000000000000000000 P dadea2b0ec8d490eb68a73fe407f3e90 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:31:38.444550 19396 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dadea2b0ec8d490eb68a73fe407f3e90 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:31:38.444708 19396 raft_consensus.cc:686] T 00000000000000000000000000000000 P dadea2b0ec8d490eb68a73fe407f3e90 [term 1 LEADER]: Becoming Leader. State: Replica: dadea2b0ec8d490eb68a73fe407f3e90, State: Running, Role: LEADER
I20260806 06:31:38.444722 19389 sys_catalog.cc:564] T 00000000000000000000000000000000 P dadea2b0ec8d490eb68a73fe407f3e90 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:31:38.444851 19396 consensus_queue.cc:237] T 00000000000000000000000000000000 P dadea2b0ec8d490eb68a73fe407f3e90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dadea2b0ec8d490eb68a73fe407f3e90" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 38047 } }
I20260806 06:31:38.445188 19399 sys_catalog.cc:455] T 00000000000000000000000000000000 P dadea2b0ec8d490eb68a73fe407f3e90 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dadea2b0ec8d490eb68a73fe407f3e90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dadea2b0ec8d490eb68a73fe407f3e90" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 38047 } } }
I20260806 06:31:38.445324 19399 sys_catalog.cc:458] T 00000000000000000000000000000000 P dadea2b0ec8d490eb68a73fe407f3e90 [sys.catalog]: This master's current role is: LEADER
I20260806 06:31:38.445573 19408 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:31:38.446069 19408 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:31:38.446125 19397 sys_catalog.cc:455] T 00000000000000000000000000000000 P dadea2b0ec8d490eb68a73fe407f3e90 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dadea2b0ec8d490eb68a73fe407f3e90. Latest consensus state: current_term: 1 leader_uuid: "dadea2b0ec8d490eb68a73fe407f3e90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dadea2b0ec8d490eb68a73fe407f3e90" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 38047 } } }
I20260806 06:31:38.446281 19397 sys_catalog.cc:458] T 00000000000000000000000000000000 P dadea2b0ec8d490eb68a73fe407f3e90 [sys.catalog]: This master's current role is: LEADER
I20260806 06:31:38.447515 19408 catalog_manager.cc:1347] Generated new cluster ID: 174921bb245a4aa5b2ef7d63502af85e
I20260806 06:31:38.447568 19408 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:31:38 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997898, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:31:38.453104 19395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:38.442179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60060 (local address 127.7.43.126:38047)
0806 06:31:38.442393 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:38.442402 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:38.442436 (+ 34us) server_negotiation.cc:407] Connection header received
0806 06:31:38.442793 (+ 357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:38.442801 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:38.442862 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:38.443007 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:38.443696 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:38.445019 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:38.446009 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:38.446619 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:38.446742 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:38.449834 (+ 3092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:38.449839 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:38.449839 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:38.449849 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:38.451606 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:38.451988 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:38.451991 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:38.451991 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:38.452095 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:38.452300 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:38.452301 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:38.452301 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:38.452559 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:38.452710 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:38.452808 (+ 98us) server_negotiation.cc:299] Negotiation successful
0806 06:31:38.452929 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":76,"threads_started":1}
I20260806 06:31:38.474427 19408 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:31:38.474895 19408 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:31:38.483247 19408 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dadea2b0ec8d490eb68a73fe407f3e90: Generated new TSK 0
I20260806 06:31:38.483402 19408 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.43.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab.
Entry for principal kudu/127.7.43.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab.
I20260806 06:31:38.506862 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.65:0
--local_ip_for_outbound_sockets=127.7.43.65
--webserver_interface=127.7.43.65
--webserver_port=0
--tserver_master_addrs=127.7.43.126:38047
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab
--principal=kudu/127.7.43.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:37075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:31:38.586508 19429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:38.586689 19429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:38.586710 19429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:38.587819 19429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:38.587859 19429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:38.587896 19429 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.65
I20260806 06:31:38.589298 19429 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:37075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab
--principal=kudu/127.7.43.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.43.65
--webserver_port=0
--tserver_master_addrs=127.7.43.126:38047
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19429
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.65
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:38.589516 19429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:38.589712 19429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:38.594238 19441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:38.594250 19438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:38.594276 19439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:38.594484 19429 server_base.cc:1034] running on GCE node
I20260806 06:31:38.594643 19429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:38.594828 19429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:38.595945 19429 hybrid_clock.cc:648] HybridClock initialized: now 1785997898595937 us; error 26 us; skew 500 ppm
Aug 06 06:31:38 dist-test-slave-6zbq krb5kdc[19273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997898, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:38.598163 19429 init.cc:377] Logged in from keytab as kudu/127.7.43.65@KRBTEST.COM (short username kudu)
I20260806 06:31:38.598582 19429 webserver.cc:460] Webserver started at http://127.7.43.65:37981/ using document root <none> and password file <none>
I20260806 06:31:38.598789 19429 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:38.598831 19429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:38.598937 19429 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:38.599854 19429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-0/data/instance:
uuid: "5108a8b17acc4a39b3c765b5f37d125b"
format_stamp: "Formatted at 2026-08-06 06:31:38 on dist-test-slave-6zbq"
server_key: "6f933a71b2a4c7ab1f7a5504cc1f3ed0"
server_key_iv: "8ace441b891e165b2dc2b36b57b8e2a9"
server_key_version: "encryptionkey@0"
I20260806 06:31:38.600159 19429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-0/wal/instance:
uuid: "5108a8b17acc4a39b3c765b5f37d125b"
format_stamp: "Formatted at 2026-08-06 06:31:38 on dist-test-slave-6zbq"
server_key: "6f933a71b2a4c7ab1f7a5504cc1f3ed0"
server_key_iv: "8ace441b891e165b2dc2b36b57b8e2a9"
server_key_version: "encryptionkey@0"
I20260806 06:31:38.601372 19429 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:31:38.601977 19454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:38.602146 19429 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:31:38.602243 19429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-0/wal
uuid: "5108a8b17acc4a39b3c765b5f37d125b"
format_stamp: "Formatted at 2026-08-06 06:31:38 on dist-test-slave-6zbq"
server_key: "6f933a71b2a4c7ab1f7a5504cc1f3ed0"
server_key_iv: "8ace441b891e165b2dc2b36b57b8e2a9"
server_key_version: "encryptionkey@0"
I20260806 06:31:38.602296 19429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:38.617002 19429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:38.618907 19429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:38.619010 19429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:38.619174 19429 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:31:38.619406 19429 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:31:38.619434 19429 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:38.619452 19429 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:31:38.619468 19429 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:38.624631 19429 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.65:37705
I20260806 06:31:38.624672 19627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.65:37705 every 8 connection(s)
I20260806 06:31:38.624930 19429 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
Aug 06 06:31:38 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997898, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.65@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:31:38.631256 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 19429
I20260806 06:31:38.631366 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-0/wal/instance
I20260806 06:31:38.631585 7341 external_mini_cluster.cc:1428] Setting key 45b9105b988eed8135507f2ee63514fa
I20260806 06:31:38.633221 19395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:38.625553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:53181 (local address 127.7.43.126:38047)
0806 06:31:38.625607 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:38.625609 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:38.625795 (+ 186us) server_negotiation.cc:407] Connection header received
0806 06:31:38.626234 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:38.626235 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:38.626248 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:38.626296 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:38.627149 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:38.627643 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:38.628500 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:38.628697 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:38.628739 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:38.630619 (+ 1880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:38.630621 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:38.630622 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:38.630632 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:38.631996 (+ 1364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:38.632367 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:38.632368 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:38.632369 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:38.632422 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:38.632673 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:38.632673 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:38.632674 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:38.632770 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:38.632815 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:38.633090 (+ 275us) server_negotiation.cc:299] Negotiation successful
0806 06:31:38.633157 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 06:31:38.633427 19631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:38.625524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38047 (local address 127.7.43.65:53181)
0806 06:31:38.625684 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:38.625716 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:38.626158 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:38.626366 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:38.626370 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:38.626901 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:38.627075 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:38.627080 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:38.627706 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:38.627707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:38.628442 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:38.628444 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:38.628747 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:38.628748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:38.628843 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:38.629207 (+ 364us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:38.629214 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:38.630530 (+ 1316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:38.632065 (+ 1535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:38.632072 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:38.632074 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:38.632297 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:38.632479 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:38.632480 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:38.632480 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:38.632613 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:38.632819 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:38.632825 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:38.632973 (+ 148us) client_negotiation.cc:769] Sending connection context
0806 06:31:38.633078 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 06:31:38.633223 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":97,"threads_started":1}
I20260806 06:31:38.633857 19628 heartbeater.cc:347] Connected to a master server at 127.7.43.126:38047
I20260806 06:31:38.633944 19628 heartbeater.cc:464] Registering TS with master...
I20260806 06:31:38.634106 19628 heartbeater.cc:511] Master 127.7.43.126:38047 requested a full tablet report, sending...
I20260806 06:31:38.634610 19330 ts_manager.cc:194] Registered new tserver with Master: 5108a8b17acc4a39b3c765b5f37d125b (127.7.43.65:37705)
I20260806 06:31:38.635174 19330 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.65@KRBTEST.COM'} at 127.7.43.65:53181
WARNING: no policy specified for kudu/127.7.43.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab.
Entry for principal kudu/127.7.43.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab.
I20260806 06:31:38.654716 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.66:0
--local_ip_for_outbound_sockets=127.7.43.66
--webserver_interface=127.7.43.66
--webserver_port=0
--tserver_master_addrs=127.7.43.126:38047
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab
--principal=kudu/127.7.43.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:37075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:31:38.732968 19634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:38.733157 19634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:38.733188 19634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:38.734298 19634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:38.734357 19634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:38.734398 19634 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.66
I20260806 06:31:38.735798 19634 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:37075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab
--principal=kudu/127.7.43.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.43.66
--webserver_port=0
--tserver_master_addrs=127.7.43.126:38047
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19634
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.66
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:38.736013 19634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:38.736217 19634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:38.740737 19641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:38.740801 19634 server_base.cc:1034] running on GCE node
W20260806 06:31:38.740716 19640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:38.740737 19643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:38.741178 19634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:38.741384 19634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:38.742511 19634 hybrid_clock.cc:648] HybridClock initialized: now 1785997898742496 us; error 32 us; skew 500 ppm
Aug 06 06:31:38 dist-test-slave-6zbq krb5kdc[19273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997898, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:38.744736 19634 init.cc:377] Logged in from keytab as kudu/127.7.43.66@KRBTEST.COM (short username kudu)
I20260806 06:31:38.745147 19634 webserver.cc:460] Webserver started at http://127.7.43.66:33415/ using document root <none> and password file <none>
I20260806 06:31:38.745393 19634 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:38.745442 19634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:38.745563 19634 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:38.746578 19634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-1/data/instance:
uuid: "88c25c6036dc42c6a10e0c0df1ed97b3"
format_stamp: "Formatted at 2026-08-06 06:31:38 on dist-test-slave-6zbq"
server_key: "d7f2643b30000bf164209b32608324f7"
server_key_iv: "850bad94f4f7da405fcf877ed6a62153"
server_key_version: "encryptionkey@0"
I20260806 06:31:38.746889 19634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-1/wal/instance:
uuid: "88c25c6036dc42c6a10e0c0df1ed97b3"
format_stamp: "Formatted at 2026-08-06 06:31:38 on dist-test-slave-6zbq"
server_key: "d7f2643b30000bf164209b32608324f7"
server_key_iv: "850bad94f4f7da405fcf877ed6a62153"
server_key_version: "encryptionkey@0"
I20260806 06:31:38.748026 19634 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:31:38.748739 19654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:38.748874 19634 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:31:38.748955 19634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-1/wal
uuid: "88c25c6036dc42c6a10e0c0df1ed97b3"
format_stamp: "Formatted at 2026-08-06 06:31:38 on dist-test-slave-6zbq"
server_key: "d7f2643b30000bf164209b32608324f7"
server_key_iv: "850bad94f4f7da405fcf877ed6a62153"
server_key_version: "encryptionkey@0"
I20260806 06:31:38.749022 19634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:38.761240 19634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:38.763199 19634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:38.763325 19634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:38.763501 19634 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:31:38.763756 19634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:31:38.763789 19634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:38.763813 19634 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:31:38.763830 19634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:38.769057 19634 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.66:44877
I20260806 06:31:38.769126 19838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.66:44877 every 8 connection(s)
I20260806 06:31:38.769366 19634 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
Aug 06 06:31:38 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997898, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.66@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:31:38.777376 19395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:38.770048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:50371 (local address 127.7.43.126:38047)
0806 06:31:38.770119 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:38.770122 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:38.770289 (+ 167us) server_negotiation.cc:407] Connection header received
0806 06:31:38.770756 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:38.770758 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:38.770770 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:38.770820 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:38.771655 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:38.772152 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:38.773066 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:38.773276 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:38.773334 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:38.775171 (+ 1837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:38.775172 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:38.775173 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:38.775180 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:38.776212 (+ 1032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:38.776546 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:38.776547 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:38.776548 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:38.776604 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:38.776809 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:38.776810 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:38.776811 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:38.776922 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:38.776977 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:38.777208 (+ 231us) server_negotiation.cc:299] Negotiation successful
0806 06:31:38.777289 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 06:31:38.777509 19842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:38.769999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38047 (local address 127.7.43.66:50371)
0806 06:31:38.770165 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:38.770217 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:38.770661 (+ 444us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:38.770881 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:38.770885 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:38.771391 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:38.771560 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:38.771565 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:38.772236 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:38.772237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:38.772980 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:38.772982 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:38.773347 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:38.773348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:38.773465 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:38.773859 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:38.773867 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:38.775084 (+ 1217us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:38.776282 (+ 1198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:38.776288 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:38.776290 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:38.776482 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:38.776656 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:38.776657 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:38.776657 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:38.776754 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:38.776975 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:38.776982 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:38.777099 (+ 117us) client_negotiation.cc:769] Sending connection context
0806 06:31:38.777180 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 06:31:38.777323 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":85,"threads_started":1}
I20260806 06:31:38.777918 19840 heartbeater.cc:347] Connected to a master server at 127.7.43.126:38047
I20260806 06:31:38.778004 19840 heartbeater.cc:464] Registering TS with master...
I20260806 06:31:38.778167 19840 heartbeater.cc:511] Master 127.7.43.126:38047 requested a full tablet report, sending...
I20260806 06:31:38.778537 19330 ts_manager.cc:194] Registered new tserver with Master: 88c25c6036dc42c6a10e0c0df1ed97b3 (127.7.43.66:44877)
I20260806 06:31:38.778817 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 19634
I20260806 06:31:38.778910 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-1/wal/instance
I20260806 06:31:38.778896 19330 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.66@KRBTEST.COM'} at 127.7.43.66:50371
I20260806 06:31:38.779106 7341 external_mini_cluster.cc:1428] Setting key fdd84e111a2a21db4e0ab1184aa90edd
WARNING: no policy specified for kudu/127.7.43.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab.
Entry for principal kudu/127.7.43.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab.
I20260806 06:31:38.801054 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.67:0
--local_ip_for_outbound_sockets=127.7.43.67
--webserver_interface=127.7.43.67
--webserver_port=0
--tserver_master_addrs=127.7.43.126:38047
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab
--principal=kudu/127.7.43.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:37075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:31:38.878393 19848 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:38.878566 19848 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:38.878595 19848 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:38.879686 19848 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:38.879731 19848 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:38.879765 19848 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.67
I20260806 06:31:38.881152 19848 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:37075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab
--principal=kudu/127.7.43.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.43.67
--webserver_port=0
--tserver_master_addrs=127.7.43.126:38047
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19848
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.67
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:38.881353 19848 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:38.881546 19848 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:38.886102 19856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:38.886123 19859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:38.886102 19853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:38.886157 19848 server_base.cc:1034] running on GCE node
I20260806 06:31:38.886494 19848 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:38.886706 19848 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:38.887831 19848 hybrid_clock.cc:648] HybridClock initialized: now 1785997898887822 us; error 26 us; skew 500 ppm
Aug 06 06:31:38 dist-test-slave-6zbq krb5kdc[19273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997898, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:38.890026 19848 init.cc:377] Logged in from keytab as kudu/127.7.43.67@KRBTEST.COM (short username kudu)
I20260806 06:31:38.890411 19848 webserver.cc:460] Webserver started at http://127.7.43.67:39999/ using document root <none> and password file <none>
I20260806 06:31:38.890575 19848 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:38.890614 19848 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:38.890705 19848 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:38.891561 19848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-2/data/instance:
uuid: "7b9cb1081e0d48608a284a2aaf625b15"
format_stamp: "Formatted at 2026-08-06 06:31:38 on dist-test-slave-6zbq"
server_key: "0759b3271b0a1207b26c4a3cf06388fe"
server_key_iv: "9c62f4bb58d6bbe2929a8dad1ae1e595"
server_key_version: "encryptionkey@0"
I20260806 06:31:38.891842 19848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-2/wal/instance:
uuid: "7b9cb1081e0d48608a284a2aaf625b15"
format_stamp: "Formatted at 2026-08-06 06:31:38 on dist-test-slave-6zbq"
server_key: "0759b3271b0a1207b26c4a3cf06388fe"
server_key_iv: "9c62f4bb58d6bbe2929a8dad1ae1e595"
server_key_version: "encryptionkey@0"
I20260806 06:31:38.892923 19848 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:31:38.893497 19871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:38.893638 19848 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:31:38.893716 19848 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-2/wal
uuid: "7b9cb1081e0d48608a284a2aaf625b15"
format_stamp: "Formatted at 2026-08-06 06:31:38 on dist-test-slave-6zbq"
server_key: "0759b3271b0a1207b26c4a3cf06388fe"
server_key_iv: "9c62f4bb58d6bbe2929a8dad1ae1e595"
server_key_version: "encryptionkey@0"
I20260806 06:31:38.893770 19848 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:38.916796 19848 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:38.918761 19848 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:38.918864 19848 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:38.919019 19848 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:31:38.919258 19848 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:31:38.919287 19848 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:38.919306 19848 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:31:38.919322 19848 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:38.924641 19848 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.67:35247
I20260806 06:31:38.924708 20048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.67:35247 every 8 connection(s)
I20260806 06:31:38.924950 19848 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
Aug 06 06:31:38 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997898, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.67@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:31:38.932680 19395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:38.925571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:54251 (local address 127.7.43.126:38047)
0806 06:31:38.925627 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:38.925629 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:38.925813 (+ 184us) server_negotiation.cc:407] Connection header received
0806 06:31:38.926288 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:38.926289 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:38.926298 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:38.926386 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:38.927219 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:38.927720 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:38.928593 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:38.928769 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:38.928810 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:38.930641 (+ 1831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:38.930642 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:38.930643 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:38.930650 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:38.931670 (+ 1020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:38.932011 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:38.932012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:38.932012 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:38.932059 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:38.932230 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:38.932231 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:38.932231 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:38.932328 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:38.932374 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:38.932562 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 06:31:38.932619 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 06:31:38.932799 20053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:38.925532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38047 (local address 127.7.43.67:54251)
0806 06:31:38.925701 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:38.925740 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:38.926211 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:38.926449 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:38.926452 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:38.926973 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:38.927148 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:38.927153 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:38.927783 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:38.927784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:38.928535 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:38.928537 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:38.928819 (+ 282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:38.928820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:38.928907 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:38.929247 (+ 340us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:38.929254 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:38.930552 (+ 1298us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:38.931747 (+ 1195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:38.931753 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:38.931756 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:38.931954 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:38.932101 (+ 147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:38.932102 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:38.932102 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:38.932184 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:38.932371 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:38.932373 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:38.932472 (+ 99us) client_negotiation.cc:769] Sending connection context
0806 06:31:38.932543 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 06:31:38.932647 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":90,"threads_started":1}
I20260806 06:31:38.933250 20051 heartbeater.cc:347] Connected to a master server at 127.7.43.126:38047
I20260806 06:31:38.933328 20051 heartbeater.cc:464] Registering TS with master...
I20260806 06:31:38.933511 20051 heartbeater.cc:511] Master 127.7.43.126:38047 requested a full tablet report, sending...
I20260806 06:31:38.933859 19330 ts_manager.cc:194] Registered new tserver with Master: 7b9cb1081e0d48608a284a2aaf625b15 (127.7.43.67:35247)
I20260806 06:31:38.934172 19330 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.67@KRBTEST.COM'} at 127.7.43.67:54251
I20260806 06:31:38.935001 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 19848
I20260806 06:31:38.935092 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-2/wal/instance
I20260806 06:31:38.935247 7341 external_mini_cluster.cc:1428] Setting key 2d73990d3120382d98466016da49a2d4
I20260806 06:31:38.935988 7341 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.7.43.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.43.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.43.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/oryx_127.7.43.65.keytab.
Entry for principal oryx/127.7.43.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/oryx_127.7.43.65.keytab.
WARNING: no policy specified for oryx/127.7.43.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.43.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.43.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/oryx_127.7.43.66.keytab.
Entry for principal oryx/127.7.43.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/oryx_127.7.43.66.keytab.
WARNING: no policy specified for oryx/127.7.43.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.43.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.43.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/oryx_127.7.43.67.keytab.
Entry for principal oryx/127.7.43.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/oryx_127.7.43.67.keytab.
I20260806 06:31:38.998005 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 19429
I20260806 06:31:39.007434 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 19634
I20260806 06:31:39.023816 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 19848
I20260806 06:31:39.034439 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 19290
I20260806 06:31:39.041000 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:38047
--webserver_interface=127.7.43.126
--webserver_port=43139
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:37075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.43.126:38047
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:31:39.118754 20063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:39.118935 20063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:39.118955 20063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:39.120069 20063 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:31:39.120106 20063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:39.120121 20063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:39.120143 20063 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:31:39.120167 20063 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:31:39.121620 20063 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:37075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.43.126:38047
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:38047
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.43.126
--webserver_port=43139
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20063
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:39.121851 20063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:39.122040 20063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:39.126397 20078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:39.126410 20075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:39.126452 20076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:39.126533 20063 server_base.cc:1034] running on GCE node
I20260806 06:31:39.126821 20063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:39.127084 20063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:39.128217 20063 hybrid_clock.cc:648] HybridClock initialized: now 1785997899128207 us; error 32 us; skew 500 ppm
Aug 06 06:31:39 dist-test-slave-6zbq krb5kdc[19273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997899, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:39.130453 20063 init.cc:377] Logged in from keytab as kudu/127.7.43.126@KRBTEST.COM (short username kudu)
I20260806 06:31:39.130880 20063 webserver.cc:460] Webserver started at http://127.7.43.126:43139/ using document root <none> and password file <none>
I20260806 06:31:39.131085 20063 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:39.131142 20063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:39.132406 20063 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:31:39.132997 20086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:39.133134 20063 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:31:39.133214 20063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "dadea2b0ec8d490eb68a73fe407f3e90"
format_stamp: "Formatted at 2026-08-06 06:31:38 on dist-test-slave-6zbq"
server_key: "ce7f8de951ec3f0141299614ca1efb88"
server_key_iv: "041c5b1dad3d90507d6ad3e4855e94fb"
server_key_version: "encryptionkey@0"
I20260806 06:31:39.133507 20063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:39.145383 20063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:39.147068 20063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:39.147185 20063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:39.151023 20063 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.126:38047
I20260806 06:31:39.151053 20164 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.126:38047 every 8 connection(s)
I20260806 06:31:39.151371 20063 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:31:39.152662 20165 sys_catalog.cc:263] Verifying existing consensus state
I20260806 06:31:39.153223 20165 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dadea2b0ec8d490eb68a73fe407f3e90: Bootstrap starting.
I20260806 06:31:39.155239 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 20063
I20260806 06:31:39.155567 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.65:37705
--local_ip_for_outbound_sockets=127.7.43.65
--tserver_master_addrs=127.7.43.126:38047
--webserver_port=37981
--webserver_interface=127.7.43.65
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab
--principal=kudu/127.7.43.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:37075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.43.65
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/oryx_127.7.43.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:31:39.161829 20165 log.cc:824] T 00000000000000000000000000000000 P dadea2b0ec8d490eb68a73fe407f3e90: Log is configured to *not* fsync() on all Append() calls
I20260806 06:31:39.163365 20165 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dadea2b0ec8d490eb68a73fe407f3e90: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:31:39.163633 20165 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dadea2b0ec8d490eb68a73fe407f3e90: Bootstrap complete.
I20260806 06:31:39.165602 20165 raft_consensus.cc:348] T 00000000000000000000000000000000 P dadea2b0ec8d490eb68a73fe407f3e90 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dadea2b0ec8d490eb68a73fe407f3e90" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 38047 } }
I20260806 06:31:39.165812 20165 raft_consensus.cc:729] T 00000000000000000000000000000000 P dadea2b0ec8d490eb68a73fe407f3e90 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dadea2b0ec8d490eb68a73fe407f3e90, State: Initialized, Role: FOLLOWER
I20260806 06:31:39.165942 20165 consensus_queue.cc:260] T 00000000000000000000000000000000 P dadea2b0ec8d490eb68a73fe407f3e90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dadea2b0ec8d490eb68a73fe407f3e90" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 38047 } }
I20260806 06:31:39.166004 20165 raft_consensus.cc:388] T 00000000000000000000000000000000 P dadea2b0ec8d490eb68a73fe407f3e90 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:31:39.166033 20165 raft_consensus.cc:482] T 00000000000000000000000000000000 P dadea2b0ec8d490eb68a73fe407f3e90 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:31:39.166076 20165 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dadea2b0ec8d490eb68a73fe407f3e90 [term 1 FOLLOWER]: Advancing to term 2
I20260806 06:31:39.167029 20165 raft_consensus.cc:504] T 00000000000000000000000000000000 P dadea2b0ec8d490eb68a73fe407f3e90 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dadea2b0ec8d490eb68a73fe407f3e90" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 38047 } }
I20260806 06:31:39.167121 20165 leader_election.cc:302] T 00000000000000000000000000000000 P dadea2b0ec8d490eb68a73fe407f3e90 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dadea2b0ec8d490eb68a73fe407f3e90; no voters:
I20260806 06:31:39.167279 20165 leader_election.cc:288] T 00000000000000000000000000000000 P dadea2b0ec8d490eb68a73fe407f3e90 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 06:31:39.167330 20170 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dadea2b0ec8d490eb68a73fe407f3e90 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 06:31:39.167523 20170 raft_consensus.cc:686] T 00000000000000000000000000000000 P dadea2b0ec8d490eb68a73fe407f3e90 [term 2 LEADER]: Becoming Leader. State: Replica: dadea2b0ec8d490eb68a73fe407f3e90, State: Running, Role: LEADER
I20260806 06:31:39.167658 20170 consensus_queue.cc:237] T 00000000000000000000000000000000 P dadea2b0ec8d490eb68a73fe407f3e90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dadea2b0ec8d490eb68a73fe407f3e90" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 38047 } }
I20260806 06:31:39.167527 20165 sys_catalog.cc:564] T 00000000000000000000000000000000 P dadea2b0ec8d490eb68a73fe407f3e90 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:31:39.167955 20171 sys_catalog.cc:455] T 00000000000000000000000000000000 P dadea2b0ec8d490eb68a73fe407f3e90 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "dadea2b0ec8d490eb68a73fe407f3e90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dadea2b0ec8d490eb68a73fe407f3e90" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 38047 } } }
I20260806 06:31:39.168038 20171 sys_catalog.cc:458] T 00000000000000000000000000000000 P dadea2b0ec8d490eb68a73fe407f3e90 [sys.catalog]: This master's current role is: LEADER
I20260806 06:31:39.168234 20174 sys_catalog.cc:455] T 00000000000000000000000000000000 P dadea2b0ec8d490eb68a73fe407f3e90 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dadea2b0ec8d490eb68a73fe407f3e90. Latest consensus state: current_term: 2 leader_uuid: "dadea2b0ec8d490eb68a73fe407f3e90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dadea2b0ec8d490eb68a73fe407f3e90" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 38047 } } }
I20260806 06:31:39.168299 20174 sys_catalog.cc:458] T 00000000000000000000000000000000 P dadea2b0ec8d490eb68a73fe407f3e90 [sys.catalog]: This master's current role is: LEADER
I20260806 06:31:39.168401 20176 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:31:39.168890 20176 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:31:39.169188 20176 catalog_manager.cc:1259] Loaded cluster ID: 174921bb245a4aa5b2ef7d63502af85e
I20260806 06:31:39.169255 20176 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:31:39.170469 20176 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:31:39.170630 20176 catalog_manager.cc:5879] T 00000000000000000000000000000000 P dadea2b0ec8d490eb68a73fe407f3e90: Loaded TSK: 0
I20260806 06:31:39.170840 20176 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 06:31:39.235765 20168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:39.235932 20168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:39.235953 20168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:39.237087 20168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:39.237125 20168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:39.237149 20168 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.65
I20260806 06:31:39.238582 20168 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:37075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/oryx_127.7.43.65.keytab
--principal=oryx/127.7.43.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.65:37705
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.43.65
--webserver_port=37981
--tserver_master_addrs=127.7.43.126:38047
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20168
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.65
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:39.238772 20168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:39.238973 20168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:39.244498 20203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:39.244509 20202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:39.244742 20168 server_base.cc:1034] running on GCE node
W20260806 06:31:39.244837 20207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:39.245040 20168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:39.245313 20168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:39.246480 20168 hybrid_clock.cc:648] HybridClock initialized: now 1785997899246461 us; error 31 us; skew 500 ppm
Aug 06 06:31:39 dist-test-slave-6zbq krb5kdc[19273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997899, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.43.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:39.249325 20168 init.cc:377] Logged in from keytab as oryx/127.7.43.65@KRBTEST.COM (short username oryx)
I20260806 06:31:39.249845 20168 webserver.cc:460] Webserver started at http://127.7.43.65:37981/ using document root <none> and password file <none>
I20260806 06:31:39.250114 20168 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:39.250196 20168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:39.251878 20168 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:31:39.252509 20217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:39.252676 20168 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:31:39.252758 20168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-0/wal
uuid: "5108a8b17acc4a39b3c765b5f37d125b"
format_stamp: "Formatted at 2026-08-06 06:31:38 on dist-test-slave-6zbq"
server_key: "6f933a71b2a4c7ab1f7a5504cc1f3ed0"
server_key_iv: "8ace441b891e165b2dc2b36b57b8e2a9"
server_key_version: "encryptionkey@0"
I20260806 06:31:39.253057 20168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:39.273052 20168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:39.274920 20168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:39.275025 20168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:39.275226 20168 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:31:39.275496 20168 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:31:39.275527 20168 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:39.275547 20168 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:31:39.275568 20168 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:39.280857 20168 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.65:37705
I20260806 06:31:39.280931 20387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.65:37705 every 8 connection(s)
I20260806 06:31:39.281211 20168 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
Aug 06 06:31:39 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.65@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:31:39 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.65@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:31:39.289624 20390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:39.281878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38047 (local address 127.7.43.65:33673)
0806 06:31:39.282031 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:39.282072 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:39.282674 (+ 602us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:39.283050 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:39.283054 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:39.283740 (+ 686us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:39.284018 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:39.284025 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:39.285018 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:39.285019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:39.286194 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:39.286197 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:39.286737 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:39.286738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:39.286864 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:39.287257 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:39.287265 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:39.289440 (+ 2175us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.43.126:38047: Server oryx/127.7.43.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":79,"threads_started":1}
W20260806 06:31:39.289664 20393 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:39.281991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:33673 (local address 127.7.43.126:38047)
0806 06:31:39.282141 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:39.282149 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:39.282178 (+ 29us) server_negotiation.cc:407] Connection header received
0806 06:31:39.282842 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:39.282848 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:39.282887 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:39.282986 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:39.284124 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:39.284922 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:39.286270 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:39.286671 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:39.286748 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:39.289548 (+ 2800us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.43.65:33673: BlockingRecv error: recv got EOF from 127.7.43.65:33673 (error 108)
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":67,"threads_started":1}
W20260806 06:31:39.289820 20388 heartbeater.cc:644] Failed to heartbeat to 127.7.43.126:38047 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.43.126:38047: Client connection negotiation failed: client connection to 127.7.43.126:38047: Server oryx/127.7.43.126@KRBTEST.COM not found in Kerberos database
I20260806 06:31:39.291057 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 20168
I20260806 06:31:39.291335 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.66:44877
--local_ip_for_outbound_sockets=127.7.43.66
--tserver_master_addrs=127.7.43.126:38047
--webserver_port=33415
--webserver_interface=127.7.43.66
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab
--principal=kudu/127.7.43.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:37075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.43.66
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/oryx_127.7.43.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 06:31:39 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.65@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:31:39 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.65@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:31:39.294476 20394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:39.289992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38047 (local address 127.7.43.65:38297)
0806 06:31:39.290114 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:39.290122 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:39.290174 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:39.290292 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:39.290293 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:39.290453 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:39.290492 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:39.290493 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:39.291150 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:39.291151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:39.291866 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:39.291869 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:39.292284 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:39.292285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:39.292338 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:39.292681 (+ 343us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:39.292686 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:39.294416 (+ 1730us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.43.126:38047: Server oryx/127.7.43.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":69,"threads_started":1}
W20260806 06:31:39.294543 20393 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:39.290029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:38297 (local address 127.7.43.126:38047)
0806 06:31:39.290070 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:39.290072 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:39.290171 (+ 99us) server_negotiation.cc:407] Connection header received
0806 06:31:39.290219 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:39.290220 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:39.290230 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:39.290284 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:39.290555 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:39.291059 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:39.291956 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:39.292191 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:39.292264 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:39.294492 (+ 2228us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.43.65:38297: BlockingRecv error: recv got EOF from 127.7.43.65:38297 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 06 06:31:39 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.65@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:31:39 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.65@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:31:39.298637 20390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:39.294683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38047 (local address 127.7.43.65:58065)
0806 06:31:39.294717 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:39.294725 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:39.294755 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:39.294880 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:39.294881 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:39.295004 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:39.295040 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:39.295041 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:39.295818 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:39.295819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:39.296290 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:39.296292 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:39.296698 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:39.296698 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:39.296743 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:39.297063 (+ 320us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:39.297067 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:39.298566 (+ 1499us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.43.126:38047: Server oryx/127.7.43.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":22}
W20260806 06:31:39.298741 20393 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:39.294709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:58065 (local address 127.7.43.126:38047)
0806 06:31:39.294738 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:39.294740 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:39.294747 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:31:39.294800 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:39.294800 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:39.294809 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:39.294859 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:39.295102 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:39.295734 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:39.296361 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:39.296632 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:39.296693 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:39.298660 (+ 1967us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.43.65:58065: BlockingRecv error: recv got EOF from 127.7.43.65:58065 (error 108)
Metrics: {"server-negotiator.queue_time_us":12}
W20260806 06:31:39.298872 20388 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 06:31:39.370949 20395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:39.371125 20395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:39.371146 20395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:39.372215 20395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:39.372246 20395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:39.372272 20395 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.66
I20260806 06:31:39.373589 20395 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:37075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/oryx_127.7.43.66.keytab
--principal=oryx/127.7.43.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.66:44877
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.43.66
--webserver_port=33415
--tserver_master_addrs=127.7.43.126:38047
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20395
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.66
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:39.373745 20395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:39.373900 20395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:39.378322 20407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:39.378401 20410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:39.378379 20406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:39.378505 20395 server_base.cc:1034] running on GCE node
I20260806 06:31:39.378723 20395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:39.378928 20395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:39.380049 20395 hybrid_clock.cc:648] HybridClock initialized: now 1785997899380035 us; error 33 us; skew 500 ppm
Aug 06 06:31:39 dist-test-slave-6zbq krb5kdc[19273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997899, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.43.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:39.382241 20395 init.cc:377] Logged in from keytab as oryx/127.7.43.66@KRBTEST.COM (short username oryx)
I20260806 06:31:39.382640 20395 webserver.cc:460] Webserver started at http://127.7.43.66:33415/ using document root <none> and password file <none>
I20260806 06:31:39.382851 20395 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:39.382906 20395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:39.384073 20395 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:31:39.384543 20420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:39.384691 20395 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:39.384760 20395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-1/wal
uuid: "88c25c6036dc42c6a10e0c0df1ed97b3"
format_stamp: "Formatted at 2026-08-06 06:31:38 on dist-test-slave-6zbq"
server_key: "d7f2643b30000bf164209b32608324f7"
server_key_iv: "850bad94f4f7da405fcf877ed6a62153"
server_key_version: "encryptionkey@0"
I20260806 06:31:39.385031 20395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:39.392427 20395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:39.394394 20395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:39.394515 20395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:39.394711 20395 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:31:39.394959 20395 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:31:39.394989 20395 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:39.395016 20395 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:31:39.395038 20395 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:39.400585 20395 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.66:44877
I20260806 06:31:39.400637 20584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.66:44877 every 8 connection(s)
I20260806 06:31:39.400913 20395 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
Aug 06 06:31:39 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.66@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:31:39 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.66@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:31:39.407016 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 20395
I20260806 06:31:39.407264 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.67:35247
--local_ip_for_outbound_sockets=127.7.43.67
--tserver_master_addrs=127.7.43.126:38047
--webserver_port=39999
--webserver_interface=127.7.43.67
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab
--principal=kudu/127.7.43.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:37075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.43.67
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/oryx_127.7.43.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:31:39.407351 20589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:39.401487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38047 (local address 127.7.43.66:54689)
0806 06:31:39.401623 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:39.401653 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:39.402075 (+ 422us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:39.402297 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:39.402301 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:39.402853 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:39.403022 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:39.403026 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:39.403680 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:39.403681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:39.404426 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:39.404428 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:39.404774 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:39.404774 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:39.404884 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:39.405229 (+ 345us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:39.405237 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:39.407173 (+ 1936us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.43.126:38047: Server oryx/127.7.43.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":73,"threads_started":1}
W20260806 06:31:39.407370 20393 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:39.401491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:54689 (local address 127.7.43.126:38047)
0806 06:31:39.401538 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:39.401541 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:39.401734 (+ 193us) server_negotiation.cc:407] Connection header received
0806 06:31:39.402169 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:39.402171 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:39.402186 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:39.402236 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:39.403112 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:39.403594 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:39.404506 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:39.404707 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:39.404761 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:39.407318 (+ 2557us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.43.66:54689: BlockingRecv error: recv got EOF from 127.7.43.66:54689 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260806 06:31:39.407769 20585 heartbeater.cc:644] Failed to heartbeat to 127.7.43.126:38047 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.43.126:38047: Client connection negotiation failed: client connection to 127.7.43.126:38047: Server oryx/127.7.43.126@KRBTEST.COM not found in Kerberos database
Aug 06 06:31:39 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.66@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:31:39 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.66@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:31:39.411597 20589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:39.407975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38047 (local address 127.7.43.66:35367)
0806 06:31:39.408013 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:39.408022 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:39.408051 (+ 29us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:39.408165 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:39.408166 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:39.408291 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:39.408338 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:39.408339 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:39.408888 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:39.408889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:39.409359 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:39.409360 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:39.409662 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:39.409663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:39.409722 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:39.410037 (+ 315us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:39.410041 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:39.411538 (+ 1497us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.43.126:38047: Server oryx/127.7.43.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":25}
W20260806 06:31:39.411661 20393 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:39.408019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:35367 (local address 127.7.43.126:38047)
0806 06:31:39.408061 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:39.408063 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:39.408071 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:31:39.408090 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:39.408091 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:39.408097 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:39.408149 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:39.408395 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:39.408818 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:39.409429 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:39.409605 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:39.409659 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:39.411608 (+ 1949us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.43.66:35367: BlockingRecv error: recv got EOF from 127.7.43.66:35367 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 06 06:31:39 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.66@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:31:39 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.66@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:31:39.415649 20589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:39.411820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38047 (local address 127.7.43.66:57219)
0806 06:31:39.411855 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:39.411864 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:39.411896 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:39.412007 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:39.412008 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:39.412131 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:39.412161 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:39.412162 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:39.412706 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:39.412706 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:39.413423 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:39.413426 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:39.413725 (+ 299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:39.413726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:39.413779 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:39.414077 (+ 298us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:39.414081 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:39.415586 (+ 1505us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.43.126:38047: Server oryx/127.7.43.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":24}
W20260806 06:31:39.415709 20393 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:39.411892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:57219 (local address 127.7.43.126:38047)
0806 06:31:39.411929 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:39.411930 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:39.411936 (+ 6us) server_negotiation.cc:407] Connection header received
0806 06:31:39.411953 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:39.411953 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:39.411960 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:39.412003 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:39.412232 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:39.412642 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:39.413498 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:39.413666 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:39.413709 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:39.415663 (+ 1954us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.43.66:57219: BlockingRecv error: recv got EOF from 127.7.43.66:57219 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260806 06:31:39.415799 20585 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 06:31:39.488631 20591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:39.488811 20591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:39.488840 20591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:39.489931 20591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:39.489969 20591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:39.490005 20591 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.67
I20260806 06:31:39.491477 20591 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:37075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/krb5kdc/oryx_127.7.43.67.keytab
--principal=oryx/127.7.43.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.67:35247
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.43.67
--webserver_port=39999
--tserver_master_addrs=127.7.43.126:38047
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20591
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.67
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:39.491704 20591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:39.491884 20591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:39.496320 20599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:39.496343 20600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:39.496363 20591 server_base.cc:1034] running on GCE node
W20260806 06:31:39.496347 20603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:39.496680 20591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:39.496893 20591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:39.498029 20591 hybrid_clock.cc:648] HybridClock initialized: now 1785997899498015 us; error 26 us; skew 500 ppm
Aug 06 06:31:39 dist-test-slave-6zbq krb5kdc[19273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997899, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.43.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:39.500176 20591 init.cc:377] Logged in from keytab as oryx/127.7.43.67@KRBTEST.COM (short username oryx)
I20260806 06:31:39.500576 20591 webserver.cc:460] Webserver started at http://127.7.43.67:39999/ using document root <none> and password file <none>
I20260806 06:31:39.500754 20591 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:39.500813 20591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:39.502017 20591 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:31:39.502604 20616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:39.502745 20591 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:31:39.502823 20591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-2/wal
uuid: "7b9cb1081e0d48608a284a2aaf625b15"
format_stamp: "Formatted at 2026-08-06 06:31:38 on dist-test-slave-6zbq"
server_key: "0759b3271b0a1207b26c4a3cf06388fe"
server_key_iv: "9c62f4bb58d6bbe2929a8dad1ae1e595"
server_key_version: "encryptionkey@0"
I20260806 06:31:39.503088 20591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:39.518597 20591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:39.520085 20591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:39.520208 20591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:39.520390 20591 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:31:39.520632 20591 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:31:39.520664 20591 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:39.520689 20591 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:31:39.520706 20591 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:39.526275 20591 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.67:35247
I20260806 06:31:39.526311 20789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.67:35247 every 8 connection(s)
I20260806 06:31:39.526615 20591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
Aug 06 06:31:39 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.67@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:31:39 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.67@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:31:39.532629 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 20591
W20260806 06:31:39.533068 20393 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:39.527346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:40161 (local address 127.7.43.126:38047)
0806 06:31:39.527397 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:39.527399 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:39.527590 (+ 191us) server_negotiation.cc:407] Connection header received
0806 06:31:39.528007 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:39.528008 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:39.528017 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:39.528067 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:39.528870 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:39.529349 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:39.530233 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:39.530470 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:39.530515 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:39.533007 (+ 2492us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.43.67:40161: BlockingRecv error: recv got EOF from 127.7.43.67:40161 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 06:31:39.533113 20792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:39.527341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38047 (local address 127.7.43.67:40161)
0806 06:31:39.527481 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:39.527518 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:39.527935 (+ 417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:39.528114 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:39.528117 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:39.528615 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:39.528790 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:39.528794 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:39.529426 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:39.529427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:39.530174 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:39.530176 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:39.530531 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:39.530532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:39.530622 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:39.530964 (+ 342us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:39.530972 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:39.532929 (+ 1957us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.43.126:38047: Server oryx/127.7.43.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":64,"threads_started":1}
W20260806 06:31:39.533306 20790 heartbeater.cc:644] Failed to heartbeat to 127.7.43.126:38047 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.43.126:38047: Client connection negotiation failed: client connection to 127.7.43.126:38047: Server oryx/127.7.43.126@KRBTEST.COM not found in Kerberos database
Aug 06 06:31:39 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.67@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:31:39 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.67@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:31:39.537563 20792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:39.533497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38047 (local address 127.7.43.67:44391)
0806 06:31:39.533532 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:39.533543 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:39.533576 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:39.533723 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:39.533723 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:39.533850 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:39.533901 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:39.533903 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:39.534508 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:39.534509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:39.535087 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:39.535089 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:39.535537 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:39.535538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:39.535599 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:39.535883 (+ 284us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:39.535888 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:39.537511 (+ 1623us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.43.126:38047: Server oryx/127.7.43.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":19}
W20260806 06:31:39.537664 20794 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:39.533525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:44391 (local address 127.7.43.126:38047)
0806 06:31:39.533611 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:39.533612 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:39.533619 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:31:39.533646 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:39.533647 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:39.533657 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:39.533707 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:39.533961 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:39.534444 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:39.535162 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:39.535475 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:39.535538 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:39.537602 (+ 2064us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.43.67:44391: BlockingRecv error: recv got EOF from 127.7.43.67:44391 (error 108)
Metrics: {"server-negotiator.queue_time_us":70,"thread_start_us":51,"threads_started":1}
I20260806 06:31:39.540031 20393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:39.533126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60072 (local address 127.7.43.126:38047)
0806 06:31:39.533181 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:39.533182 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:39.533266 (+ 84us) server_negotiation.cc:407] Connection header received
0806 06:31:39.533367 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:39.533368 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:39.533377 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:39.533434 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:39.534133 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:39.534710 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:39.535578 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:39.535829 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:39.535892 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:39.537178 (+ 1286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:39.537183 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:39.537183 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:39.537195 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:39.538866 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:39.539160 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:39.539162 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:39.539163 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:39.539279 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:39.539441 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:39.539442 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:39.539442 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:39.539633 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:39.539753 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:39.539860 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 06:31:39.539952 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":7,"server-negotiator.queue_time_us":38}
Aug 06 06:31:39 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.67@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:31:39 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.67@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:31:39.541606 20792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:39.537790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38047 (local address 127.7.43.67:45759)
0806 06:31:39.537826 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:39.537834 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:39.537875 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:39.537992 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:39.537993 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:39.538093 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:39.538120 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:39.538122 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:39.538713 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:39.538714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:39.539338 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:39.539341 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:39.539683 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:39.539684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:39.539732 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:39.540099 (+ 367us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:39.540105 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:39.541569 (+ 1464us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.43.126:38047: Server oryx/127.7.43.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":25}
W20260806 06:31:39.541687 20794 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:39.537804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:45759 (local address 127.7.43.126:38047)
0806 06:31:39.537830 (+ 26us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:39.537832 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:39.537864 (+ 32us) server_negotiation.cc:407] Connection header received
0806 06:31:39.537918 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:39.537918 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:39.537925 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:39.537987 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:39.538183 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:39.538646 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:39.539406 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:39.539618 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:39.539664 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:39.541640 (+ 1976us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.43.67:45759: BlockingRecv error: recv got EOF from 127.7.43.67:45759 (error 108)
Metrics: {"server-negotiator.queue_time_us":12}
W20260806 06:31:39.541723 20790 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 06 06:31:40 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.65@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:31:40 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.65@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:31:40.303318 20795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:40.299169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38047 (local address 127.7.43.65:48577)
0806 06:31:40.299280 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:40.299290 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:40.299339 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:40.299455 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:40.299455 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:40.299597 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:40.299644 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:40.299645 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:40.300240 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:40.300241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:40.300758 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:40.300759 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:40.301029 (+ 270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:40.301030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:40.301064 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:40.301567 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:40.301572 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:40.303256 (+ 1684us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.43.126:38047: Server oryx/127.7.43.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":63,"threads_started":1}
W20260806 06:31:40.303414 20798 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:40.299255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:48577 (local address 127.7.43.126:38047)
0806 06:31:40.299353 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:40.299355 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:40.299362 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:31:40.299381 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:40.299382 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:40.299391 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:40.299445 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:40.299694 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:40.300182 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:40.300811 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:40.300985 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:40.301027 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:40.303350 (+ 2323us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.43.65:48577: BlockingRecv error: recv got EOF from 127.7.43.65:48577 (error 108)
Metrics: {"server-negotiator.queue_time_us":80,"thread_start_us":54,"threads_started":1}
Aug 06 06:31:40 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.66@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:31:40 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.66@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:31:40.420161 20799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:40.416108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38047 (local address 127.7.43.66:34343)
0806 06:31:40.416227 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:40.416237 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:40.416289 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:40.416396 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:40.416397 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:40.416541 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:40.416588 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:40.416589 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:40.417151 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:40.417152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:40.417675 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:40.417677 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:40.418004 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:40.418005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:40.418061 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:40.418511 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:40.418516 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:40.420096 (+ 1580us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.43.126:38047: Server oryx/127.7.43.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":53,"threads_started":1}
W20260806 06:31:40.420228 20798 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:40.416167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:34343 (local address 127.7.43.126:38047)
0806 06:31:40.416221 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:40.416223 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:40.416279 (+ 56us) server_negotiation.cc:407] Connection header received
0806 06:31:40.416329 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:40.416329 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:40.416338 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:40.416385 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:40.416633 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:40.417102 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:40.417760 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:40.417944 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:40.417999 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:40.420177 (+ 2178us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.43.66:34343: BlockingRecv error: recv got EOF from 127.7.43.66:34343 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 06 06:31:40 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.67@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:31:40 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.67@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:31:40.546590 20800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:40.542086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38047 (local address 127.7.43.67:34935)
0806 06:31:40.542212 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:40.542222 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:40.542285 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:40.542432 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:40.542433 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:40.542579 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:40.542640 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:40.542642 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:40.543253 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:40.543254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:40.543798 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:40.543800 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:40.544151 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:40.544152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:40.544207 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:40.544813 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:40.544818 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:40.546505 (+ 1687us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.43.126:38047: Server oryx/127.7.43.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":55,"threads_started":1}
W20260806 06:31:40.546653 20798 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:40.542168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:34935 (local address 127.7.43.126:38047)
0806 06:31:40.542216 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:40.542218 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:40.542280 (+ 62us) server_negotiation.cc:407] Connection header received
0806 06:31:40.542356 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:40.542357 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:40.542371 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:40.542422 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:40.542694 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:40.543184 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:40.543886 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:40.544075 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:40.544148 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:40.546597 (+ 2449us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.43.67:34935: BlockingRecv error: recv got EOF from 127.7.43.67:34935 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 06 06:31:41 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.65@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:31:41 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.65@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:31:41.310652 20801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:41.303718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38047 (local address 127.7.43.65:37293)
0806 06:31:41.303850 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:41.303860 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:41.303909 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:41.304054 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:41.304054 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:41.304212 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:41.304261 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:41.304263 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:41.304856 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:41.304857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:41.305375 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:41.305376 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:41.305711 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:41.305712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:41.305755 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:41.309027 (+ 3272us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:41.309032 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:41.310596 (+ 1564us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.43.126:38047: Server oryx/127.7.43.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":81,"threads_started":1}
W20260806 06:31:41.310761 20802 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:41.303811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:37293 (local address 127.7.43.126:38047)
0806 06:31:41.303904 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:41.303906 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:41.303913 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:31:41.303971 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:41.303972 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:41.303982 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:41.304039 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:41.304308 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:41.304804 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:41.305431 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:41.305655 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:41.305709 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:41.310693 (+ 4984us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.43.65:37293: BlockingRecv error: recv got EOF from 127.7.43.65:37293 (error 108)
Metrics: {"server-negotiator.queue_time_us":77,"thread_start_us":48,"threads_started":1}
Aug 06 06:31:41 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.66@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:31:41 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.66@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:31:41.424965 20805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:41.420582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38047 (local address 127.7.43.66:34385)
0806 06:31:41.420725 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:41.420736 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:41.420800 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:41.420920 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:41.420921 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:41.421084 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:41.421157 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:41.421159 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:41.421786 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:41.421787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:41.422310 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:41.422311 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:41.422839 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:41.422841 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:41.422901 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:41.423313 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:41.423318 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:41.424887 (+ 1569us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.43.126:38047: Server oryx/127.7.43.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":78,"threads_started":1}
W20260806 06:31:41.425035 20802 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:41.420651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:34385 (local address 127.7.43.126:38047)
0806 06:31:41.420711 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:41.420713 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:41.420782 (+ 69us) server_negotiation.cc:407] Connection header received
0806 06:31:41.420849 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:41.420850 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:41.420861 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:41.420903 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:41.421230 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:41.421715 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:41.422482 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:41.422770 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:41.422838 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:41.424983 (+ 2145us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.43.66:34385: BlockingRecv error: recv got EOF from 127.7.43.66:34385 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 06 06:31:41 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.67@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:31:41 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.67@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:31:41.551088 20806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:41.546981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38047 (local address 127.7.43.67:39821)
0806 06:31:41.547117 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:41.547129 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:41.547184 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:41.547297 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:41.547298 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:41.547442 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:41.547489 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:41.547491 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:41.548082 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:41.548082 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:41.548588 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:41.548589 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:41.548858 (+ 269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:41.548859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:41.548891 (+ 32us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:41.549251 (+ 360us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:41.549255 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:41.551025 (+ 1770us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.43.126:38047: Server oryx/127.7.43.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":79,"threads_started":1}
W20260806 06:31:41.551241 20802 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:41.547107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:39821 (local address 127.7.43.126:38047)
0806 06:31:41.547172 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:41.547174 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:41.547182 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:31:41.547238 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:41.547239 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:41.547250 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:41.547287 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:41.547538 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:41.548016 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:41.548635 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:41.548813 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:41.548855 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:41.551177 (+ 2322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.43.67:39821: BlockingRecv error: recv got EOF from 127.7.43.67:39821 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 06 06:31:42 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.65@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:31:42 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.65@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:31:42.315438 20808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:42.311073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38047 (local address 127.7.43.65:34619)
0806 06:31:42.311254 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:42.311265 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:42.311334 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:42.311465 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:42.311466 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:42.311620 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:42.311669 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:42.311671 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:42.312298 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:42.312300 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:42.312899 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:42.312902 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:42.313233 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:42.313234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:42.313276 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:42.313709 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:42.313713 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:42.315378 (+ 1665us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.43.126:38047: Server oryx/127.7.43.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":92,"threads_started":1}
W20260806 06:31:42.315572 20810 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:42.311176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:34619 (local address 127.7.43.126:38047)
0806 06:31:42.311285 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:42.311288 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:42.311316 (+ 28us) server_negotiation.cc:407] Connection header received
0806 06:31:42.311384 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:42.311385 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:42.311396 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:42.311453 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:42.311722 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:42.312215 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:42.312965 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:42.313175 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:42.313227 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:42.315500 (+ 2273us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.43.65:34619: BlockingRecv error: recv got EOF from 127.7.43.65:34619 (error 108)
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":53,"threads_started":1}
Aug 06 06:31:42 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.66@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:31:42 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.66@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:31:42.429829 20811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:42.425451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38047 (local address 127.7.43.66:58141)
0806 06:31:42.425602 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:42.425617 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:42.425690 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:42.425825 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:42.425826 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:42.425997 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:42.426042 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:42.426043 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:42.426717 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:42.426718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:42.427252 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:42.427255 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:42.427572 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:42.427573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:42.427627 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:42.428019 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:42.428023 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:42.429746 (+ 1723us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.43.126:38047: Server oryx/127.7.43.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":84,"threads_started":1}
W20260806 06:31:42.429901 20810 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:42.425554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:58141 (local address 127.7.43.126:38047)
0806 06:31:42.425631 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:42.425633 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:42.425669 (+ 36us) server_negotiation.cc:407] Connection header received
0806 06:31:42.425751 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:42.425751 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:42.425766 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:42.425812 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:42.426110 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:42.426646 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:42.427330 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:42.427514 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:42.427568 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:42.429843 (+ 2275us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.43.66:58141: BlockingRecv error: recv got EOF from 127.7.43.66:58141 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 06 06:31:42 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.67@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:31:42 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.67@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:31:42.555424 20813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:42.551469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38047 (local address 127.7.43.67:58749)
0806 06:31:42.551620 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:42.551629 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:42.551682 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:42.551799 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:42.551799 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:42.551944 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:42.551989 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:42.551991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:42.552544 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:42.552545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:42.553057 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:42.553059 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:42.553327 (+ 268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:42.553328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:42.553366 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:42.553740 (+ 374us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:42.553744 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:42.555359 (+ 1615us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.43.126:38047: Server oryx/127.7.43.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":80,"threads_started":1}
W20260806 06:31:42.555503 20810 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:42.551535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:58749 (local address 127.7.43.126:38047)
0806 06:31:42.551587 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:42.551589 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:42.551683 (+ 94us) server_negotiation.cc:407] Connection header received
0806 06:31:42.551734 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:42.551735 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:42.551746 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:42.551782 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:42.552039 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:42.552494 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:42.553111 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:42.553278 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:42.553319 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:42.555443 (+ 2124us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.43.67:58749: BlockingRecv error: recv got EOF from 127.7.43.67:58749 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 06 06:31:43 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.65@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:31:43 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.65@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:31:43.320092 20816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:43.315898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38047 (local address 127.7.43.65:43799)
0806 06:31:43.316034 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:43.316044 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:43.316095 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:43.316220 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:43.316220 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:43.316368 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:43.316419 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:43.316420 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:43.317087 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:43.317088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:43.317628 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:43.317629 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:43.317906 (+ 277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:43.317907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:43.317943 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:43.318372 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:43.318378 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:43.320035 (+ 1657us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.43.126:38047: Server oryx/127.7.43.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":65,"threads_started":1}
W20260806 06:31:43.320186 20817 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:43.315995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:43799 (local address 127.7.43.126:38047)
0806 06:31:43.316083 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:43.316085 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:43.316093 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:31:43.316139 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:43.316140 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:43.316150 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:43.316210 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:43.316468 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:43.317016 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:43.317681 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:43.317862 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:43.317903 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:43.320120 (+ 2217us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.43.65:43799: BlockingRecv error: recv got EOF from 127.7.43.65:43799 (error 108)
Metrics: {"server-negotiator.queue_time_us":71,"thread_start_us":43,"threads_started":1}
Aug 06 06:31:43 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.66@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:31:43 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.66@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:31:43.434464 20818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:43.430245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38047 (local address 127.7.43.66:57725)
0806 06:31:43.430395 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:43.430406 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:43.430474 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:43.430608 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:43.430609 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:43.430781 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:43.430828 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:43.430829 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:43.431467 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:43.431469 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:43.431983 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:43.431985 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:43.432303 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:43.432304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:43.432356 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:43.432757 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:43.432761 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:43.434391 (+ 1630us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.43.126:38047: Server oryx/127.7.43.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":69,"threads_started":1}
W20260806 06:31:43.434520 20817 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:43.430311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:57725 (local address 127.7.43.126:38047)
0806 06:31:43.430388 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:43.430390 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:43.430465 (+ 75us) server_negotiation.cc:407] Connection header received
0806 06:31:43.430533 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:43.430534 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:43.430546 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:43.430598 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:43.430897 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:43.431394 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:43.432058 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:43.432244 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:43.432299 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:43.434470 (+ 2171us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.43.66:57725: BlockingRecv error: recv got EOF from 127.7.43.66:57725 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 06 06:31:43 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.67@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:31:43 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.67@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:31:43.560065 20819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:43.555816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38047 (local address 127.7.43.67:51431)
0806 06:31:43.555933 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:43.555944 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:43.556011 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:43.556136 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:43.556137 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:43.556306 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:43.556351 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:43.556353 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:43.556989 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:43.556991 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:43.557504 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:43.557506 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:43.557838 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:43.557839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:43.557895 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:43.558296 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:43.558300 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:43.559989 (+ 1689us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.43.126:38047: Server oryx/127.7.43.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":66,"threads_started":1}
W20260806 06:31:43.560132 20817 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:43.555884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:51431 (local address 127.7.43.126:38047)
0806 06:31:43.555944 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:43.555946 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:43.555990 (+ 44us) server_negotiation.cc:407] Connection header received
0806 06:31:43.556065 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:43.556066 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:43.556078 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:43.556121 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:43.556415 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:43.556921 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:43.557580 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:43.557778 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:43.557832 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:43.560081 (+ 2249us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.43.67:51431: BlockingRecv error: recv got EOF from 127.7.43.67:51431 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 06 06:31:44 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.65@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:31:44 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.65@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:31:44.324877 20821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:44.320539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38047 (local address 127.7.43.65:58601)
0806 06:31:44.320657 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:44.320669 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:44.320733 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:44.320864 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:44.320865 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:44.321013 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:44.321064 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:44.321065 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:44.321677 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:44.321677 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:44.322216 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:44.322217 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:44.322567 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:44.322568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:44.322607 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:44.323126 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:44.323131 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:44.324820 (+ 1689us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.43.126:38047: Server oryx/127.7.43.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":57,"threads_started":1}
W20260806 06:31:44.324965 20822 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:44.320615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:58601 (local address 127.7.43.126:38047)
0806 06:31:44.320747 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:44.320749 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:44.320758 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:31:44.320780 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:44.320780 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:44.320791 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:44.320854 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:44.321129 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:44.321622 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:44.322281 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:44.322517 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:44.322568 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:44.324902 (+ 2334us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.43.65:58601: BlockingRecv error: recv got EOF from 127.7.43.65:58601 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":64,"threads_started":1}
Aug 06 06:31:44 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.66@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:31:44 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.66@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:31:44.439139 20827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:44.434887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38047 (local address 127.7.43.66:40733)
0806 06:31:44.435035 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:44.435047 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:44.435108 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:44.435223 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:44.435224 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:44.435373 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:44.435422 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:44.435424 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:44.436022 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:44.436023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:44.436600 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:44.436602 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:44.436888 (+ 286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:44.436889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:44.436924 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:44.437344 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:44.437348 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:44.439053 (+ 1705us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.43.126:38047: Server oryx/127.7.43.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":84,"threads_started":1}
W20260806 06:31:44.439196 20822 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:44.435010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:40733 (local address 127.7.43.126:38047)
0806 06:31:44.435062 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:44.435064 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:44.435093 (+ 29us) server_negotiation.cc:407] Connection header received
0806 06:31:44.435157 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:44.435157 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:44.435167 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:44.435212 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:44.435465 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:44.435970 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:44.436656 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:44.436840 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:44.436885 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:44.439144 (+ 2259us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.43.66:40733: BlockingRecv error: recv got EOF from 127.7.43.66:40733 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 06 06:31:44 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.67@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:31:44 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.67@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:31:44.564736 20828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:44.560482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38047 (local address 127.7.43.67:60027)
0806 06:31:44.560635 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:44.560646 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:44.560712 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:44.560844 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:44.560845 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:44.561009 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:44.561056 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:44.561058 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:44.561696 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:44.561697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:44.562215 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:44.562216 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:44.562576 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:44.562577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:44.562635 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:44.563071 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:44.563076 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:44.564661 (+ 1585us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.43.126:38047: Server oryx/127.7.43.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":83,"threads_started":1}
W20260806 06:31:44.564812 20822 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:44.560546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:60027 (local address 127.7.43.126:38047)
0806 06:31:44.560612 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:44.560614 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:44.560702 (+ 88us) server_negotiation.cc:407] Connection header received
0806 06:31:44.560769 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:44.560769 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:44.560783 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:44.560831 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:44.561133 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:44.561617 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:44.562292 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:44.562520 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:44.562576 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:44.564758 (+ 2182us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.43.67:60027: BlockingRecv error: recv got EOF from 127.7.43.67:60027 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 06 06:31:45 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.65@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:31:45 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.65@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:31:45.329236 20832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:45.325300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38047 (local address 127.7.43.65:38107)
0806 06:31:45.325413 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:45.325424 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:45.325485 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:45.325604 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:45.325605 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:45.325750 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:45.325796 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:45.325798 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:45.326424 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:45.326424 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:45.326931 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:45.326932 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:45.327235 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:45.327235 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:45.327270 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:45.327636 (+ 366us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:45.327640 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:45.329181 (+ 1541us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.43.126:38047: Server oryx/127.7.43.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":59,"threads_started":1}
W20260806 06:31:45.329327 20833 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:45.325357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:38107 (local address 127.7.43.126:38047)
0806 06:31:45.325496 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:45.325498 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:45.325505 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:31:45.325525 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:45.325526 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:45.325536 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:45.325593 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:45.325845 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:45.326366 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:45.326988 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:45.327189 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:45.327232 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:45.329259 (+ 2027us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.43.65:38107: BlockingRecv error: recv got EOF from 127.7.43.65:38107 (error 108)
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":82,"threads_started":1}
Aug 06 06:31:45 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.66@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:31:45 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.66@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:31:45.443668 20835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:45.439598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38047 (local address 127.7.43.66:44633)
0806 06:31:45.439715 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:45.439725 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:45.439779 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:45.439891 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:45.439892 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:45.440038 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:45.440085 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:45.440086 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:45.440696 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:45.440697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:45.441286 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:45.441289 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:45.441645 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:45.441646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:45.441701 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:45.442078 (+ 377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:45.442082 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:45.443613 (+ 1531us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.43.126:38047: Server oryx/127.7.43.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":53,"threads_started":1}
W20260806 06:31:45.443748 20833 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:45.439647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:44633 (local address 127.7.43.126:38047)
0806 06:31:45.439698 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:45.439700 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:45.439775 (+ 75us) server_negotiation.cc:407] Connection header received
0806 06:31:45.439832 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:45.439833 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:45.439842 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:45.439880 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:45.440129 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:45.440630 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:45.441353 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:45.441583 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:45.441634 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:45.443696 (+ 2062us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.43.66:44633: BlockingRecv error: recv got EOF from 127.7.43.66:44633 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 06 06:31:45 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.67@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:31:45 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.67@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:31:45.569209 20836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:45.565160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38047 (local address 127.7.43.67:45323)
0806 06:31:45.565279 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:45.565290 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:45.565346 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:45.565448 (+ 102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:45.565448 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:45.565593 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:45.565639 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:45.565641 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:45.566227 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:45.566227 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:45.566786 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:45.566789 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:45.567051 (+ 262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:45.567052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:45.567087 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:45.567522 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:45.567526 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:45.569152 (+ 1626us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.43.126:38047: Server oryx/127.7.43.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":66,"threads_started":1}
W20260806 06:31:45.569293 20833 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:45.565219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:45323 (local address 127.7.43.126:38047)
0806 06:31:45.565294 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:45.565295 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:45.565332 (+ 37us) server_negotiation.cc:407] Connection header received
0806 06:31:45.565394 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:45.565395 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:45.565406 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:45.565443 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:45.565692 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:45.566175 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:45.566839 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:45.567006 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:45.567047 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:45.569230 (+ 2183us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.43.67:45323: BlockingRecv error: recv got EOF from 127.7.43.67:45323 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 06 06:31:46 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.65@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:31:46 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.65@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:31:46.336807 20838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:46.329662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38047 (local address 127.7.43.65:59823)
0806 06:31:46.329792 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:46.329805 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:46.329864 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:46.329986 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:46.329987 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:46.330139 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:46.330187 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:46.330189 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:46.330809 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:46.330810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:46.331338 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:46.331340 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:46.331653 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:46.331654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:46.331690 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:46.334948 (+ 3258us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:46.334953 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:46.336749 (+ 1796us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.43.126:38047: Server oryx/127.7.43.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":61,"threads_started":1}
W20260806 06:31:46.336913 20841 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:46.329724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:59823 (local address 127.7.43.126:38047)
0806 06:31:46.329855 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:46.329857 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:46.329865 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:31:46.329908 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:46.329909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:46.329919 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:46.329974 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:46.330239 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:46.330743 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:46.331409 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:46.331603 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:46.331650 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:46.336838 (+ 5188us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.43.65:59823: BlockingRecv error: recv got EOF from 127.7.43.65:59823 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":64,"threads_started":1}
Aug 06 06:31:46 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.66@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:31:46 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.66@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:31:46.448279 20843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:46.444072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38047 (local address 127.7.43.66:38029)
0806 06:31:46.444198 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:46.444217 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:46.444283 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:46.444416 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:46.444417 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:46.444580 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:46.444628 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:46.444629 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:46.445265 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:46.445266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:46.445786 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:46.445788 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:46.446108 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:46.446109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:46.446161 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:46.446608 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:46.446613 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:46.448217 (+ 1604us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.43.126:38047: Server oryx/127.7.43.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":65,"threads_started":1}
W20260806 06:31:46.448377 20841 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:46.444137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:38029 (local address 127.7.43.126:38047)
0806 06:31:46.444202 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:46.444205 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:46.444274 (+ 69us) server_negotiation.cc:407] Connection header received
0806 06:31:46.444344 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:46.444344 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:46.444356 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:46.444405 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:46.444700 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:46.445189 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:46.445862 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:46.446049 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:46.446105 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:46.448310 (+ 2205us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.43.66:38029: BlockingRecv error: recv got EOF from 127.7.43.66:38029 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 06 06:31:46 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.67@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:31:46 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.67@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:31:46.573817 20845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:46.569586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38047 (local address 127.7.43.67:43989)
0806 06:31:46.569719 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:46.569729 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:46.569797 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:46.569922 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:46.569923 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:46.570083 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:46.570133 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:46.570135 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:46.570808 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:46.570809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:46.571356 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:46.571358 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:46.571689 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:46.571689 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:46.571746 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:46.572189 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:46.572194 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:46.573744 (+ 1550us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.43.126:38047: Server oryx/127.7.43.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":71,"threads_started":1}
W20260806 06:31:46.573902 20841 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:46.569637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:43989 (local address 127.7.43.126:38047)
0806 06:31:46.569697 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:46.569699 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:46.569787 (+ 88us) server_negotiation.cc:407] Connection header received
0806 06:31:46.569850 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:46.569850 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:46.569864 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:46.569908 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:46.570202 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:46.570740 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:46.571435 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:46.571629 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:46.571683 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:46.573848 (+ 2165us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.43.67:43989: BlockingRecv error: recv got EOF from 127.7.43.67:43989 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 06 06:31:47 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.65@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:31:47 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.65@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:31:47.341331 20846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:47.337189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38047 (local address 127.7.43.65:37455)
0806 06:31:47.337319 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:47.337330 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:47.337376 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:47.337497 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:47.337497 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:47.337644 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:47.337694 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:47.337695 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:47.338302 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:47.338302 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:47.338836 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:47.338838 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:47.339135 (+ 297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:47.339136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:47.339170 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:47.339580 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:47.339585 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:47.341255 (+ 1670us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.43.126:38047: Server oryx/127.7.43.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":55,"threads_started":1}
W20260806 06:31:47.341457 20847 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:47.337278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:37455 (local address 127.7.43.126:38047)
0806 06:31:47.337377 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:47.337379 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:47.337407 (+ 28us) server_negotiation.cc:407] Connection header received
0806 06:31:47.337427 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:47.337428 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:47.337438 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:47.337487 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:47.337742 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:47.338249 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:47.338891 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:47.339074 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:47.339133 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:47.341356 (+ 2223us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.43.65:37455: BlockingRecv error: recv got EOF from 127.7.43.65:37455 (error 108)
Metrics: {"server-negotiator.queue_time_us":81,"thread_start_us":51,"threads_started":1}
Aug 06 06:31:47 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.66@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:31:47 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.66@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:31:47.452795 20848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:47.448653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38047 (local address 127.7.43.66:37507)
0806 06:31:47.448788 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:47.448798 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:47.448858 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:47.448968 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:47.448969 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:47.449115 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:47.449163 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:47.449164 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:47.449883 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:47.449883 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:47.450440 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:47.450442 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:47.450725 (+ 283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:47.450725 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:47.450766 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:47.451152 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:47.451156 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:47.452728 (+ 1572us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.43.126:38047: Server oryx/127.7.43.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":66,"threads_started":1}
W20260806 06:31:47.452905 20847 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:47.448698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:37507 (local address 127.7.43.126:38047)
0806 06:31:47.448744 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:47.448746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:47.448854 (+ 108us) server_negotiation.cc:407] Connection header received
0806 06:31:47.448909 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:47.448910 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:47.448921 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:47.448956 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:47.449215 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:47.449827 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:47.450496 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:47.450675 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:47.450722 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:47.452842 (+ 2120us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.43.66:37507: BlockingRecv error: recv got EOF from 127.7.43.66:37507 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Aug 06 06:31:47 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.67@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:31:47 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.67@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:31:47.578315 20850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:47.574216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38047 (local address 127.7.43.67:54117)
0806 06:31:47.574326 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:47.574357 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:47.574422 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:47.574552 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:47.574553 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:47.574714 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:47.574764 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:47.574766 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:47.575386 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:47.575387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:47.575908 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:47.575910 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:47.576231 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:47.576232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:47.576288 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:47.576704 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:47.576709 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:47.578238 (+ 1529us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.43.126:38047: Server oryx/127.7.43.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":63,"threads_started":1}
W20260806 06:31:47.578413 20847 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:47.574287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:54117 (local address 127.7.43.126:38047)
0806 06:31:47.574362 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:47.574364 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:47.574401 (+ 37us) server_negotiation.cc:407] Connection header received
0806 06:31:47.574476 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:47.574477 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:47.574491 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:47.574544 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:47.574831 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:47.575317 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:47.575980 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:47.576166 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:47.576219 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:47.578335 (+ 2116us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.43.67:54117: BlockingRecv error: recv got EOF from 127.7.43.67:54117 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 06 06:31:48 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.65@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:31:48 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.65@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:31:48.346212 20851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:48.341739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38047 (local address 127.7.43.65:37389)
0806 06:31:48.341869 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:48.341878 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:48.341932 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:48.342055 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:48.342056 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:48.342207 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:48.342255 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:48.342256 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:48.342891 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:48.342891 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:48.343609 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:48.343612 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:48.343918 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:48.343919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:48.343953 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:48.344381 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:48.344386 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:48.346099 (+ 1713us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.43.126:38047: Server oryx/127.7.43.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":62,"threads_started":1}
W20260806 06:31:48.346252 20852 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:48.341787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:37389 (local address 127.7.43.126:38047)
0806 06:31:48.341894 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:48.341895 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:48.341917 (+ 22us) server_negotiation.cc:407] Connection header received
0806 06:31:48.341977 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:48.341978 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:48.341988 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:48.342045 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:48.342312 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:48.342826 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:48.343671 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:48.343866 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:48.343922 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:48.346190 (+ 2268us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.43.65:37389: BlockingRecv error: recv got EOF from 127.7.43.65:37389 (error 108)
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":58,"threads_started":1}
Aug 06 06:31:48 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.66@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:31:48 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.66@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:31:48.457540 20853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:48.453248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38047 (local address 127.7.43.66:58609)
0806 06:31:48.453384 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:48.453395 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:48.453452 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:48.453564 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:48.453565 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:48.453717 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:48.453777 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:48.453778 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:48.454423 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:48.454424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:48.454977 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:48.454979 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:48.455363 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:48.455364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:48.455409 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:48.455803 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:48.455807 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:48.457484 (+ 1677us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.43.126:38047: Server oryx/127.7.43.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":76,"threads_started":1}
W20260806 06:31:48.457650 20852 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:48.453322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:58609 (local address 127.7.43.126:38047)
0806 06:31:48.453381 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:48.453383 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:48.453445 (+ 62us) server_negotiation.cc:407] Connection header received
0806 06:31:48.453503 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:48.453504 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:48.453515 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:48.453553 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:48.453823 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:48.454318 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:48.455051 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:48.455296 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:48.455353 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:48.457583 (+ 2230us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.43.66:58609: BlockingRecv error: recv got EOF from 127.7.43.66:58609 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 06 06:31:48 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.67@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:31:48 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.67@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:31:48.583187 20855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:48.578800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38047 (local address 127.7.43.67:39557)
0806 06:31:48.578954 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:48.578966 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:48.579044 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:48.579187 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:48.579188 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:48.579360 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:48.579405 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:48.579407 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:48.580028 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:48.580029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:48.580549 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:48.580551 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:48.580877 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:48.580878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:48.580934 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:48.581388 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:48.581394 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:48.583121 (+ 1727us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.43.126:38047: Server oryx/127.7.43.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":71,"threads_started":1}
W20260806 06:31:48.583276 20852 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:48.578863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:39557 (local address 127.7.43.126:38047)
0806 06:31:48.578952 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:48.578954 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:48.579032 (+ 78us) server_negotiation.cc:407] Connection header received
0806 06:31:48.579105 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:48.579106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:48.579120 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:48.579168 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:48.579477 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:48.579955 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:48.580637 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:48.580821 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:48.580873 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:48.583208 (+ 2335us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.43.67:39557: BlockingRecv error: recv got EOF from 127.7.43.67:39557 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 06 06:31:49 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.65@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:31:49 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.65@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:31:49.351078 20859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:49.346659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38047 (local address 127.7.43.65:58053)
0806 06:31:49.346778 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:49.346789 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:49.346837 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:49.346970 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:49.346970 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:49.347151 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:49.347214 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:49.347217 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:49.347887 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:49.347889 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:49.348609 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:49.348613 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:49.348948 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:49.348950 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:49.349007 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:49.349415 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:49.349419 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:49.351002 (+ 1583us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.43.126:38047: Server oryx/127.7.43.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":69,"threads_started":1}
W20260806 06:31:49.351161 20861 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:49.346701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:58053 (local address 127.7.43.126:38047)
0806 06:31:49.346830 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:49.346832 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:49.346841 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:31:49.346886 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:49.346886 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:49.346897 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:49.346953 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:49.347298 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:49.347808 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:49.348684 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:49.348887 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:49.348941 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:49.351099 (+ 2158us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.43.65:58053: BlockingRecv error: recv got EOF from 127.7.43.65:58053 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":69,"threads_started":1}
Aug 06 06:31:49 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.66@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:31:49 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.66@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:31:49.462129 20862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:49.457925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38047 (local address 127.7.43.66:52933)
0806 06:31:49.458050 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:49.458063 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:49.458133 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:49.458265 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:49.458265 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:49.458469 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:49.458515 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:49.458517 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:49.459136 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:49.459137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:49.459665 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:49.459667 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:49.459980 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:49.459981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:49.460038 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:49.460455 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:49.460460 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:49.462060 (+ 1600us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.43.126:38047: Server oryx/127.7.43.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":67,"threads_started":1}
W20260806 06:31:49.462293 20861 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:49.457994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:52933 (local address 127.7.43.126:38047)
0806 06:31:49.458054 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:49.458056 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:49.458119 (+ 63us) server_negotiation.cc:407] Connection header received
0806 06:31:49.458191 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:49.458191 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:49.458204 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:49.458252 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:49.458580 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:49.459068 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:49.459738 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:49.459924 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:49.459977 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:49.462240 (+ 2263us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.43.66:52933: BlockingRecv error: recv got EOF from 127.7.43.66:52933 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 06 06:31:49 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.67@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:31:49 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.67@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:31:49.587920 20863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:49.583626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38047 (local address 127.7.43.67:35239)
0806 06:31:49.583750 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:49.583762 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:49.583822 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:49.583947 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:49.583948 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:49.584118 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:49.584164 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:49.584166 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:49.584789 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:49.584791 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:49.585322 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:49.585324 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:49.585643 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:49.585644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:49.585700 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:49.586104 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:49.586107 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:49.587863 (+ 1756us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.43.126:38047: Server oryx/127.7.43.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":66,"threads_started":1}
W20260806 06:31:49.588034 20861 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:49.583758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:35239 (local address 127.7.43.126:38047)
0806 06:31:49.583801 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:49.583802 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:49.583810 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:31:49.583874 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:49.583874 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:49.583887 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:49.583940 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:49.584234 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:49.584713 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:49.585398 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:49.585586 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:49.585639 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:49.587945 (+ 2306us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.43.67:35239: BlockingRecv error: recv got EOF from 127.7.43.67:35239 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 06 06:31:50 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.65@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:31:50 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.65@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:31:50.355674 20865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:50.351547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38047 (local address 127.7.43.65:54263)
0806 06:31:50.351668 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:50.351678 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:50.351730 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:50.351847 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:50.351847 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:50.352001 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:50.352052 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:50.352054 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:50.352651 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:50.352652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:50.353184 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:50.353186 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:50.353490 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:50.353491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:50.353528 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:50.353941 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:50.353945 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:50.355616 (+ 1671us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.43.126:38047: Server oryx/127.7.43.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":68,"threads_started":1}
W20260806 06:31:50.355804 20867 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:50.351623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:54263 (local address 127.7.43.126:38047)
0806 06:31:50.351735 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:50.351736 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:50.351746 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:31:50.351768 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:50.351768 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:50.351778 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:50.351833 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:50.352101 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:50.352599 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:50.353253 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:50.353440 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:50.353488 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:50.355728 (+ 2240us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.43.65:54263: BlockingRecv error: recv got EOF from 127.7.43.65:54263 (error 108)
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":52,"threads_started":1}
Aug 06 06:31:50 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.66@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:31:50 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.66@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:31:50.466732 20868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:50.462625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38047 (local address 127.7.43.66:39275)
0806 06:31:50.462749 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:50.462759 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:50.462816 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:50.462925 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:50.462926 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:50.463099 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:50.463149 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:50.463151 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:50.463750 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:50.463750 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:50.464263 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:50.464264 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:50.464559 (+ 295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:50.464559 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:50.464597 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:50.465033 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:50.465038 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:50.466662 (+ 1624us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.43.126:38047: Server oryx/127.7.43.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":67,"threads_started":1}
W20260806 06:31:50.466816 20867 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:50.462688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:39275 (local address 127.7.43.126:38047)
0806 06:31:50.462740 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:50.462742 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:50.462801 (+ 59us) server_negotiation.cc:407] Connection header received
0806 06:31:50.462857 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:50.462858 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:50.462868 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:50.462914 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:50.463202 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:50.463693 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:50.464321 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:50.464506 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:50.464558 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:50.466755 (+ 2197us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.43.66:39275: BlockingRecv error: recv got EOF from 127.7.43.66:39275 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 06 06:31:50 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.67@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:31:50 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.67@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:31:50.592340 20869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:50.588307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38047 (local address 127.7.43.67:54905)
0806 06:31:50.588430 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:50.588442 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:50.588500 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:50.588610 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:50.588610 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:50.588772 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:50.588818 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:50.588820 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:50.589397 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:50.589397 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:50.589929 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:50.589930 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:50.590189 (+ 259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:50.590190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:50.590225 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:50.590676 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:50.590681 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:50.592272 (+ 1591us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.43.126:38047: Server oryx/127.7.43.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":64,"threads_started":1}
W20260806 06:31:50.592430 20867 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:50.588367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:54905 (local address 127.7.43.126:38047)
0806 06:31:50.588421 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:50.588422 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:50.588489 (+ 67us) server_negotiation.cc:407] Connection header received
0806 06:31:50.588549 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:50.588550 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:50.588562 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:50.588597 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:50.588867 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:50.589346 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:50.589980 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:50.590147 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:50.590196 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:50.592368 (+ 2172us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.43.67:54905: BlockingRecv error: recv got EOF from 127.7.43.67:54905 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 06 06:31:51 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.65@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:31:51 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.65@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:31:51.362888 20870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:51.356055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38047 (local address 127.7.43.65:55689)
0806 06:31:51.356189 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:51.356201 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:51.356264 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:51.356397 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:51.356397 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:51.356556 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:51.356606 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:51.356607 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:51.357195 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:51.357196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:51.357725 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:51.357726 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:51.358004 (+ 278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:51.358005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:51.358040 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:51.361246 (+ 3206us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:51.361251 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:51.362832 (+ 1581us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.43.126:38047: Server oryx/127.7.43.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":67,"threads_started":1}
W20260806 06:31:51.362994 20873 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:51.356119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:55689 (local address 127.7.43.126:38047)
0806 06:31:51.356229 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:51.356231 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:51.356243 (+ 12us) server_negotiation.cc:407] Connection header received
0806 06:31:51.356320 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:51.356320 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:51.356330 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:51.356386 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:51.356655 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:51.357137 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:51.357781 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:51.357958 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:51.358001 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:51.362919 (+ 4918us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.43.65:55689: BlockingRecv error: recv got EOF from 127.7.43.65:55689 (error 108)
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":61,"threads_started":1}
Aug 06 06:31:51 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.66@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:31:51 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.66@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:31:51.471451 20875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:51.467117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38047 (local address 127.7.43.66:35447)
0806 06:31:51.467249 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:51.467272 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:51.467347 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:51.467459 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:51.467460 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:51.467621 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:51.467666 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:51.467668 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:51.468315 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:51.468316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:51.468908 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:51.468911 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:51.469206 (+ 295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:51.469207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:51.469242 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:51.469668 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:51.469673 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:51.471362 (+ 1689us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.43.126:38047: Server oryx/127.7.43.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":72,"threads_started":1}
W20260806 06:31:51.471524 20873 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:51.467195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:35447 (local address 127.7.43.126:38047)
0806 06:31:51.467254 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:51.467256 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:51.467335 (+ 79us) server_negotiation.cc:407] Connection header received
0806 06:31:51.467393 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:51.467393 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:51.467405 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:51.467448 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:51.467713 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:51.468248 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:51.468957 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:51.469156 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:51.469205 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:51.471472 (+ 2267us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.43.66:35447: BlockingRecv error: recv got EOF from 127.7.43.66:35447 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 06 06:31:51 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.67@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:31:51 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.67@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:31:51.596933 20878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:51.592749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38047 (local address 127.7.43.67:51921)
0806 06:31:51.592865 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:51.592874 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:51.592932 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:51.593041 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:51.593041 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:51.593202 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:51.593251 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:51.593252 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:51.593888 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:51.593888 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:51.594462 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:51.594464 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:51.594774 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:51.594774 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:51.594823 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:51.595207 (+ 384us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:51.595210 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:51.596864 (+ 1654us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.43.126:38047: Server oryx/127.7.43.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":55,"threads_started":1}
W20260806 06:31:51.597003 20873 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:51.592798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:51921 (local address 127.7.43.126:38047)
0806 06:31:51.592845 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:51.592847 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:51.592923 (+ 76us) server_negotiation.cc:407] Connection header received
0806 06:31:51.592971 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:51.592972 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:51.592982 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:51.593028 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:51.593330 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:51.593823 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:51.594522 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:51.594716 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:51.594769 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:51.596942 (+ 2173us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.43.67:51921: BlockingRecv error: recv got EOF from 127.7.43.67:51921 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 06 06:31:52 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.65@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:31:52 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.65@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:31:52.367772 20880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:52.363341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38047 (local address 127.7.43.65:43383)
0806 06:31:52.363488 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:52.363499 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:52.363555 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:52.363691 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:52.363691 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:52.363852 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:52.363900 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:52.363902 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:52.364510 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:52.364512 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:52.365054 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:52.365056 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:52.365409 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:52.365410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:52.365458 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:52.365952 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:52.365957 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:52.367715 (+ 1758us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.43.126:38047: Server oryx/127.7.43.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":73,"threads_started":1}
W20260806 06:31:52.367875 20882 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:52.363417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:43383 (local address 127.7.43.126:38047)
0806 06:31:52.363523 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:52.363524 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:52.363537 (+ 13us) server_negotiation.cc:407] Connection header received
0806 06:31:52.363605 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:52.363605 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:52.363623 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:52.363679 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:52.363949 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:52.364440 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:52.365137 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:52.365338 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:52.365395 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:52.367801 (+ 2406us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.43.65:43383: BlockingRecv error: recv got EOF from 127.7.43.65:43383 (error 108)
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":49,"threads_started":1}
Aug 06 06:31:52 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.66@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:31:52 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.66@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:31:52.476181 20883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:52.471866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38047 (local address 127.7.43.66:44831)
0806 06:31:52.472008 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:52.472019 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:52.472081 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:52.472215 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:52.472215 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:52.472389 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:52.472434 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:52.472435 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:52.473061 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:52.473062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:52.473601 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:52.473603 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:52.473926 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:52.473927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:52.473986 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:52.474407 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:52.474411 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:52.476098 (+ 1687us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.43.126:38047: Server oryx/127.7.43.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":68,"threads_started":1}
W20260806 06:31:52.476272 20882 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:52.471961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:44831 (local address 127.7.43.126:38047)
0806 06:31:52.472013 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:52.472015 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:52.472063 (+ 48us) server_negotiation.cc:407] Connection header received
0806 06:31:52.472143 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:52.472144 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:52.472154 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:52.472197 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:52.472504 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:52.472989 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:52.473676 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:52.473869 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:52.473921 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:52.476214 (+ 2293us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.43.66:44831: BlockingRecv error: recv got EOF from 127.7.43.66:44831 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 06 06:31:52 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.67@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:31:52 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.67@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:31:52.601596 20884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:52.597352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38047 (local address 127.7.43.67:46801)
0806 06:31:52.597520 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:52.597530 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:52.597591 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:52.597724 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:52.597725 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:52.597890 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:52.597955 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:52.597958 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:52.598625 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:52.598626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:52.599146 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:52.599148 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:52.599460 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:52.599461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:52.599517 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:52.599930 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:52.599935 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:52.601528 (+ 1593us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.43.126:38047: Server oryx/127.7.43.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":76,"threads_started":1}
W20260806 06:31:52.601671 20882 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:52.597389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:46801 (local address 127.7.43.126:38047)
0806 06:31:52.597437 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:52.597439 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:52.597583 (+ 144us) server_negotiation.cc:407] Connection header received
0806 06:31:52.597650 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:52.597650 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:52.597663 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:52.597712 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:52.598016 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:52.598557 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:52.599220 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:52.599403 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:52.599455 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:52.601611 (+ 2156us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.43.67:46801: BlockingRecv error: recv got EOF from 127.7.43.67:46801 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 06 06:31:53 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.65@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:31:53 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.65@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:31:53.372598 20885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:53.368150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38047 (local address 127.7.43.65:55043)
0806 06:31:53.368296 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:53.368307 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:53.368360 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:53.368481 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:53.368481 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:53.368684 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:53.368736 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:53.368739 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:53.369419 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:53.369419 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:53.369932 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:53.369933 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:53.370245 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:53.370246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:53.370300 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:53.370821 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:53.370825 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:53.372523 (+ 1698us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.43.126:38047: Server oryx/127.7.43.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":64,"threads_started":1}
W20260806 06:31:53.372682 20886 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:53.368188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:55043 (local address 127.7.43.126:38047)
0806 06:31:53.368347 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:53.368349 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:53.368357 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:31:53.368412 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:53.368412 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:53.368422 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:53.368471 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:53.368806 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:53.369357 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:53.369988 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:53.370169 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:53.370216 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:53.372623 (+ 2407us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.43.65:55043: BlockingRecv error: recv got EOF from 127.7.43.65:55043 (error 108)
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":80,"threads_started":1}
Aug 06 06:31:53 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.66@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:31:53 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.66@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:31:53.480825 20887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:53.476595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38047 (local address 127.7.43.66:36013)
0806 06:31:53.476711 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:53.476722 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:53.476784 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:53.476913 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:53.476913 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:53.477083 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:53.477144 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:53.477146 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:53.477783 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:53.477784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:53.478302 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:53.478304 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:53.478719 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:53.478721 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:53.478779 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:53.479190 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:53.479195 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:53.480751 (+ 1556us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.43.126:38047: Server oryx/127.7.43.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":63,"threads_started":1}
W20260806 06:31:53.480909 20886 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:53.476653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:36013 (local address 127.7.43.126:38047)
0806 06:31:53.476714 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:53.476716 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:53.476775 (+ 59us) server_negotiation.cc:407] Connection header received
0806 06:31:53.476842 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:53.476844 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:53.476855 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:53.476899 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:53.477217 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:53.477712 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:53.478407 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:53.478652 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:53.478709 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:53.480852 (+ 2143us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.43.66:36013: BlockingRecv error: recv got EOF from 127.7.43.66:36013 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 06 06:31:53 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.67@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:31:53 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.67@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:31:53.606432 20888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:53.602015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38047 (local address 127.7.43.67:43133)
0806 06:31:53.602146 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:53.602155 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:53.602201 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:53.602321 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:53.602321 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:53.602490 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:53.602537 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:53.602538 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:53.603336 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:53.603337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:53.603870 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:53.603872 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:53.604177 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:53.604178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:53.604228 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:53.604600 (+ 372us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:53.604604 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:53.606333 (+ 1729us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.43.126:38047: Server oryx/127.7.43.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":67,"threads_started":1}
W20260806 06:31:53.606482 20886 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:53.602082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:43133 (local address 127.7.43.126:38047)
0806 06:31:53.602133 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:53.602135 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:53.602192 (+ 57us) server_negotiation.cc:407] Connection header received
0806 06:31:53.602259 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:53.602260 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:53.602270 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:53.602308 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:53.602586 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:53.603275 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:53.603930 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:53.604104 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:53.604156 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:53.606435 (+ 2279us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.43.67:43133: BlockingRecv error: recv got EOF from 127.7.43.67:43133 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 06 06:31:54 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.65@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:31:54 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.65@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:31:54.377326 20890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:54.373108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38047 (local address 127.7.43.65:45753)
0806 06:31:54.373262 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:54.373274 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:54.373339 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:54.373495 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:54.373495 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:54.373689 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:54.373751 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:54.373754 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:54.374397 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:54.374397 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:54.374909 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:54.374911 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:54.375209 (+ 298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:54.375209 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:54.375244 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:54.375677 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:54.375681 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:54.377261 (+ 1580us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.43.126:38047: Server oryx/127.7.43.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":89,"threads_started":1}
W20260806 06:31:54.377415 20891 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:54.373162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:45753 (local address 127.7.43.126:38047)
0806 06:31:54.373282 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:54.373284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:54.373331 (+ 47us) server_negotiation.cc:407] Connection header received
0806 06:31:54.373388 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:54.373388 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:54.373400 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:54.373468 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:54.373822 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:54.374322 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:54.374963 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:54.375157 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:54.375200 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:54.377349 (+ 2149us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.43.65:45753: BlockingRecv error: recv got EOF from 127.7.43.65:45753 (error 108)
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":56,"threads_started":1}
Aug 06 06:31:54 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.66@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:31:54 dist-test-slave-6zbq krb5kdc[19273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.43.66@KRBTEST.COM for oryx/127.7.43.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:31:54.485512 20892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:54.481322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38047 (local address 127.7.43.66:54805)
0806 06:31:54.481460 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:54.481471 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:54.481539 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:54.481671 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:54.481672 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:54.481835 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:54.481880 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:54.481881 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:54.482542 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:54.482542 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:54.483054 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:54.483056 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:54.483399 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:54.483399 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:54.483454 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:54.483889 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:54.483894 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:54.485425 (+ 1531us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.43.126:38047: Server oryx/127.7.43.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":82,"threads_started":1}
W20260806 06:31:54.485576 20891 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:54.481401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:54805 (local address 127.7.43.126:38047)
0806 06:31:54.481477 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:54.481478 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:54.481521 (+ 43us) server_negotiation.cc:407] Connection header received
0806 06:31:54.481601 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:54.481602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:54.481613 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:54.481660 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:54.481950 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:54.482477 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:54.483127 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:54.483336 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:54.483394 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:54.485525 (+ 2131us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.43.66:54805: BlockingRecv error: recv got EOF from 127.7.43.66:54805 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 06:31:54.541495 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 20168
I20260806 06:31:54.550726 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 20395
I20260806 06:31:54.556136 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 20591
I20260806 06:31:54.565332 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 20063
2026-08-06T06:31:54Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17460 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-06T06:31:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:31:54Z Disabled control of system clock
I20260806 06:31:54.594290 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:36375
--webserver_interface=127.7.43.126
--webserver_port=0
--builtin_ntp_servers=127.7.43.84:37581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.43.126:36375
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 06:31:54.674943 20900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:54.675129 20900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:54.675148 20900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:54.676211 20900 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:31:54.676244 20900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:54.676255 20900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:54.676266 20900 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:31:54.676282 20900 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:31:54.677536 20900 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:37581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.43.126:36375
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:36375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.43.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20900
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:54.677711 20900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:54.677927 20900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:54.682476 20918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:54.682574 20900 server_base.cc:1034] running on GCE node
W20260806 06:31:54.682516 20911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:54.682479 20908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:54.682883 20900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:54.683135 20900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:54.684276 20900 hybrid_clock.cc:648] HybridClock initialized: now 1785997914684262 us; error 27 us; skew 500 ppm
I20260806 06:31:54.685318 20900 webserver.cc:460] Webserver started at http://127.7.43.126:43363/ using document root <none> and password file <none>
I20260806 06:31:54.685514 20900 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:54.685559 20900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:54.685680 20900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:54.686671 20900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/master-0/data/instance:
uuid: "3cfd01968b3c450896a0217c9470384c"
format_stamp: "Formatted at 2026-08-06 06:31:54 on dist-test-slave-6zbq"
server_key: "48be6f7effde2a3f6523b114591c6ed7"
server_key_iv: "051ed6de0fb3d5b782f478d27b3e4094"
server_key_version: "encryptionkey@0"
I20260806 06:31:54.686975 20900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/master-0/wal/instance:
uuid: "3cfd01968b3c450896a0217c9470384c"
format_stamp: "Formatted at 2026-08-06 06:31:54 on dist-test-slave-6zbq"
server_key: "48be6f7effde2a3f6523b114591c6ed7"
server_key_iv: "051ed6de0fb3d5b782f478d27b3e4094"
server_key_version: "encryptionkey@0"
I20260806 06:31:54.688122 20900 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:31:54.688742 20926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:54.688889 20900 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:31:54.688967 20900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "3cfd01968b3c450896a0217c9470384c"
format_stamp: "Formatted at 2026-08-06 06:31:54 on dist-test-slave-6zbq"
server_key: "48be6f7effde2a3f6523b114591c6ed7"
server_key_iv: "051ed6de0fb3d5b782f478d27b3e4094"
server_key_version: "encryptionkey@0"
I20260806 06:31:54.689024 20900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:54.715551 20900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:54.715811 20900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:54.715940 20900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:54.719548 20900 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.126:36375
I20260806 06:31:54.719587 21004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.126:36375 every 8 connection(s)
I20260806 06:31:54.719870 20900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:31:54.720484 21005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:54.722895 21005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cfd01968b3c450896a0217c9470384c: Bootstrap starting.
I20260806 06:31:54.723416 21005 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3cfd01968b3c450896a0217c9470384c: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:54.723647 21005 log.cc:824] T 00000000000000000000000000000000 P 3cfd01968b3c450896a0217c9470384c: Log is configured to *not* fsync() on all Append() calls
I20260806 06:31:54.724213 21005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cfd01968b3c450896a0217c9470384c: No bootstrap required, opened a new log
I20260806 06:31:54.725664 21005 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3cfd01968b3c450896a0217c9470384c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cfd01968b3c450896a0217c9470384c" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36375 } }
I20260806 06:31:54.725790 21005 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3cfd01968b3c450896a0217c9470384c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:54.725818 21005 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3cfd01968b3c450896a0217c9470384c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cfd01968b3c450896a0217c9470384c, State: Initialized, Role: FOLLOWER
I20260806 06:31:54.725924 21005 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3cfd01968b3c450896a0217c9470384c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cfd01968b3c450896a0217c9470384c" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36375 } }
I20260806 06:31:54.725996 21005 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3cfd01968b3c450896a0217c9470384c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:31:54.726023 21005 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3cfd01968b3c450896a0217c9470384c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:31:54.726052 21005 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3cfd01968b3c450896a0217c9470384c [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:54.726542 21005 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3cfd01968b3c450896a0217c9470384c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cfd01968b3c450896a0217c9470384c" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36375 } }
I20260806 06:31:54.726625 21005 leader_election.cc:302] T 00000000000000000000000000000000 P 3cfd01968b3c450896a0217c9470384c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3cfd01968b3c450896a0217c9470384c; no voters:
I20260806 06:31:54.726752 21005 leader_election.cc:288] T 00000000000000000000000000000000 P 3cfd01968b3c450896a0217c9470384c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:31:54.726804 21012 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3cfd01968b3c450896a0217c9470384c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:31:54.726945 21005 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3cfd01968b3c450896a0217c9470384c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:31:54.726966 21012 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3cfd01968b3c450896a0217c9470384c [term 1 LEADER]: Becoming Leader. State: Replica: 3cfd01968b3c450896a0217c9470384c, State: Running, Role: LEADER
I20260806 06:31:54.727056 21012 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3cfd01968b3c450896a0217c9470384c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cfd01968b3c450896a0217c9470384c" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36375 } }
I20260806 06:31:54.727558 21015 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cfd01968b3c450896a0217c9470384c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3cfd01968b3c450896a0217c9470384c. Latest consensus state: current_term: 1 leader_uuid: "3cfd01968b3c450896a0217c9470384c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cfd01968b3c450896a0217c9470384c" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36375 } } }
I20260806 06:31:54.727648 21015 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cfd01968b3c450896a0217c9470384c [sys.catalog]: This master's current role is: LEADER
I20260806 06:31:54.727897 21013 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cfd01968b3c450896a0217c9470384c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3cfd01968b3c450896a0217c9470384c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cfd01968b3c450896a0217c9470384c" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36375 } } }
I20260806 06:31:54.727963 21025 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:31:54.728030 21013 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cfd01968b3c450896a0217c9470384c [sys.catalog]: This master's current role is: LEADER
I20260806 06:31:54.728091 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 20900
I20260806 06:31:54.728180 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/master-0/wal/instance
I20260806 06:31:54.728386 7341 external_mini_cluster.cc:1428] Setting key 62944554d5f400154f099b3e733644fd
I20260806 06:31:54.728467 21025 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:31:54.729851 21025 catalog_manager.cc:1347] Generated new cluster ID: 04be01b6c4ad4122933dd146e65ca3ac
I20260806 06:31:54.729895 21025 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:31:54.743342 21036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:54.728764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35564 (local address 127.7.43.126:36375)
0806 06:31:54.738453 (+ 9689us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:54.738459 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:54.738484 (+ 25us) server_negotiation.cc:407] Connection header received
0806 06:31:54.738570 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:54.738575 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:54.738621 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:54.738738 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:54.738986 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:54.740321 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:54.741406 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:54.741976 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:54.742077 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:54.742728 (+ 651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:54.742732 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:54.742732 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:31:54.742920 (+ 188us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:54.742957 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:31:54.742970 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:54.743028 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:54.743090 (+ 62us) server_negotiation.cc:299] Negotiation successful
0806 06:31:54.743160 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9624,"thread_start_us":55,"threads_started":1}
I20260806 06:31:54.746238 21025 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:31:54.746701 21025 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:31:54.757620 21025 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3cfd01968b3c450896a0217c9470384c: Generated new TSK 0
I20260806 06:31:54.757754 21025 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:31:54.760447 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.65:0
--local_ip_for_outbound_sockets=127.7.43.65
--webserver_interface=127.7.43.65
--webserver_port=0
--tserver_master_addrs=127.7.43.126:36375
--builtin_ntp_servers=127.7.43.84:37581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260806 06:31:54.845948 21038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:54.846130 21038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:54.846158 21038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:54.847270 21038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:54.847306 21038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:54.847338 21038 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.65
I20260806 06:31:54.848601 21038 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:37581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.43.65
--webserver_port=0
--tserver_master_addrs=127.7.43.126:36375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21038
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.65
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:54.848810 21038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:54.849058 21038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:54.853492 21048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:54.853519 21043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:54.853649 21045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:54.853725 21038 server_base.cc:1034] running on GCE node
I20260806 06:31:54.853893 21038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:54.854067 21038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:54.855188 21038 hybrid_clock.cc:648] HybridClock initialized: now 1785997914855176 us; error 28 us; skew 500 ppm
I20260806 06:31:54.856278 21038 webserver.cc:460] Webserver started at http://127.7.43.65:35241/ using document root <none> and password file <none>
I20260806 06:31:54.856498 21038 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:54.856544 21038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:54.856633 21038 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:54.857547 21038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-0/data/instance:
uuid: "14a494ce786f468f8d58087ec2cd5da5"
format_stamp: "Formatted at 2026-08-06 06:31:54 on dist-test-slave-6zbq"
server_key: "93847c881f09370ecbb74d5752496274"
server_key_iv: "ad716b39e32d627ea51d0b92619f06fd"
server_key_version: "encryptionkey@0"
I20260806 06:31:54.857820 21038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-0/wal/instance:
uuid: "14a494ce786f468f8d58087ec2cd5da5"
format_stamp: "Formatted at 2026-08-06 06:31:54 on dist-test-slave-6zbq"
server_key: "93847c881f09370ecbb74d5752496274"
server_key_iv: "ad716b39e32d627ea51d0b92619f06fd"
server_key_version: "encryptionkey@0"
I20260806 06:31:54.858976 21038 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:31:54.859570 21056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:54.859710 21038 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:31:54.859788 21038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-0/wal
uuid: "14a494ce786f468f8d58087ec2cd5da5"
format_stamp: "Formatted at 2026-08-06 06:31:54 on dist-test-slave-6zbq"
server_key: "93847c881f09370ecbb74d5752496274"
server_key_iv: "ad716b39e32d627ea51d0b92619f06fd"
server_key_version: "encryptionkey@0"
I20260806 06:31:54.859848 21038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:54.878468 21038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:54.878702 21038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:54.878816 21038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:54.879014 21038 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:31:54.879310 21038 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:31:54.879343 21038 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:54.879361 21038 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:31:54.879372 21038 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:54.884642 21038 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.65:36595
I20260806 06:31:54.884696 21215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.65:36595 every 8 connection(s)
I20260806 06:31:54.884953 21038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
I20260806 06:31:54.889822 21036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:54.885557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:40503 (local address 127.7.43.126:36375)
0806 06:31:54.885613 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:54.885616 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:54.885820 (+ 204us) server_negotiation.cc:407] Connection header received
0806 06:31:54.886291 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:54.886293 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:54.886308 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:54.886393 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:54.887093 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:54.887734 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:54.888646 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:54.888856 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:54.888910 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:54.889562 (+ 652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:54.889563 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:54.889563 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:31:54.889568 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:54.889582 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:31:54.889586 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:54.889630 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:54.889709 (+ 79us) server_negotiation.cc:299] Negotiation successful
0806 06:31:54.889737 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 06:31:54.889998 21219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:54.885523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:36375 (local address 127.7.43.65:40503)
0806 06:31:54.885717 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:54.885746 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:54.886195 (+ 449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:54.886459 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:54.886463 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:54.886473 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:54.887016 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:54.887021 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:54.887833 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:54.887834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:54.888567 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:54.888569 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:54.888924 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:54.888925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:54.889054 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:54.889442 (+ 388us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:31:54.889442 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:54.889468 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:31:54.889488 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:54.889636 (+ 148us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:54.889641 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:54.889644 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:31:54.889710 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 06:31:54.889803 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":81,"threads_started":1}
I20260806 06:31:54.890437 21216 heartbeater.cc:347] Connected to a master server at 127.7.43.126:36375
I20260806 06:31:54.890524 21216 heartbeater.cc:464] Registering TS with master...
I20260806 06:31:54.890661 21216 heartbeater.cc:511] Master 127.7.43.126:36375 requested a full tablet report, sending...
I20260806 06:31:54.891113 20952 ts_manager.cc:194] Registered new tserver with Master: 14a494ce786f468f8d58087ec2cd5da5 (127.7.43.65:36595)
I20260806 06:31:54.891760 20952 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.43.65:40503
I20260806 06:31:54.893290 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 21038
I20260806 06:31:54.893364 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-0/wal/instance
I20260806 06:31:54.893530 7341 external_mini_cluster.cc:1428] Setting key b9ae56a235231d24e19d677d7863485e
I20260806 06:31:54.894280 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.66:0
--local_ip_for_outbound_sockets=127.7.43.66
--webserver_interface=127.7.43.66
--webserver_port=0
--tserver_master_addrs=127.7.43.126:36375
--builtin_ntp_servers=127.7.43.84:37581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260806 06:31:54.972178 21221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:54.972352 21221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:54.972379 21221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:54.973462 21221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:54.973498 21221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:54.973529 21221 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.66
I20260806 06:31:54.974850 21221 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:37581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.43.66
--webserver_port=0
--tserver_master_addrs=127.7.43.126:36375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.66
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:54.975066 21221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:54.975318 21221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:54.979769 21226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:54.979889 21230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:54.979761 21227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:54.979797 21221 server_base.cc:1034] running on GCE node
I20260806 06:31:54.980149 21221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:54.980381 21221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:54.981518 21221 hybrid_clock.cc:648] HybridClock initialized: now 1785997914981509 us; error 32 us; skew 500 ppm
I20260806 06:31:54.982617 21221 webserver.cc:460] Webserver started at http://127.7.43.66:45069/ using document root <none> and password file <none>
I20260806 06:31:54.982831 21221 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:54.982872 21221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:54.982992 21221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:54.983948 21221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-1/data/instance:
uuid: "31622659b7d0469ca58500a534c4d557"
format_stamp: "Formatted at 2026-08-06 06:31:54 on dist-test-slave-6zbq"
server_key: "6aae14b291b310fb6ec80f77dfc9e5f2"
server_key_iv: "f74f1c48bb8d7d1c9ecc51237928698b"
server_key_version: "encryptionkey@0"
I20260806 06:31:54.984265 21221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-1/wal/instance:
uuid: "31622659b7d0469ca58500a534c4d557"
format_stamp: "Formatted at 2026-08-06 06:31:54 on dist-test-slave-6zbq"
server_key: "6aae14b291b310fb6ec80f77dfc9e5f2"
server_key_iv: "f74f1c48bb8d7d1c9ecc51237928698b"
server_key_version: "encryptionkey@0"
I20260806 06:31:54.985409 21221 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:31:54.986006 21241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:54.986145 21221 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:31:54.986225 21221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-1/wal
uuid: "31622659b7d0469ca58500a534c4d557"
format_stamp: "Formatted at 2026-08-06 06:31:54 on dist-test-slave-6zbq"
server_key: "6aae14b291b310fb6ec80f77dfc9e5f2"
server_key_iv: "f74f1c48bb8d7d1c9ecc51237928698b"
server_key_version: "encryptionkey@0"
I20260806 06:31:54.986295 21221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:54.998628 21221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:54.998857 21221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:54.998967 21221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:54.999147 21221 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:31:54.999428 21221 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:31:54.999460 21221 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:54.999480 21221 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:31:54.999502 21221 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:55.004882 21221 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.66:40595
I20260806 06:31:55.004921 21404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.66:40595 every 8 connection(s)
I20260806 06:31:55.005184 21221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
I20260806 06:31:55.008999 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 21221
I20260806 06:31:55.009091 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-1/wal/instance
I20260806 06:31:55.009289 7341 external_mini_cluster.cc:1428] Setting key 40843e98bb993ad144e2255df5e3cfd8
I20260806 06:31:55.009719 21036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:55.005826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:36107 (local address 127.7.43.126:36375)
0806 06:31:55.005887 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:55.005888 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:55.006077 (+ 189us) server_negotiation.cc:407] Connection header received
0806 06:31:55.006558 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:55.006559 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:55.006568 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:55.006617 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:55.007172 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:55.007647 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:55.008520 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:55.008693 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:55.008735 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:55.009475 (+ 740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:55.009476 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:55.009477 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:31:55.009482 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:55.009497 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:31:55.009501 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:55.009552 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:55.009632 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 06:31:55.009660 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 06:31:55.009918 21407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:55.005808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:36375 (local address 127.7.43.66:36107)
0806 06:31:55.005973 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:55.006010 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:55.006478 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:55.006667 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:55.006670 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:55.006678 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:55.007112 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:55.007116 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:55.007704 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:55.007704 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:55.008463 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:55.008466 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:55.008743 (+ 277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:55.008744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:55.008838 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:55.009328 (+ 490us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:31:55.009329 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:55.009365 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:31:55.009391 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:55.009560 (+ 169us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:55.009565 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:55.009567 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:31:55.009630 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 06:31:55.009726 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":74,"threads_started":1}
I20260806 06:31:55.010322 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.67:0
--local_ip_for_outbound_sockets=127.7.43.67
--webserver_interface=127.7.43.67
--webserver_port=0
--tserver_master_addrs=127.7.43.126:36375
--builtin_ntp_servers=127.7.43.84:37581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 06:31:55.010372 21405 heartbeater.cc:347] Connected to a master server at 127.7.43.126:36375
I20260806 06:31:55.010452 21405 heartbeater.cc:464] Registering TS with master...
I20260806 06:31:55.010602 21405 heartbeater.cc:511] Master 127.7.43.126:36375 requested a full tablet report, sending...
I20260806 06:31:55.010950 20952 ts_manager.cc:194] Registered new tserver with Master: 31622659b7d0469ca58500a534c4d557 (127.7.43.66:40595)
I20260806 06:31:55.011360 20952 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.43.66:36107
W20260806 06:31:55.090953 21408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:55.091140 21408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:55.091159 21408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:55.092226 21408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:55.092257 21408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:55.092279 21408 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.67
I20260806 06:31:55.093494 21408 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:37581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.43.67
--webserver_port=0
--tserver_master_addrs=127.7.43.126:36375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21408
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.67
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:55.093652 21408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:55.093873 21408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:55.098410 21415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:55.098486 21408 server_base.cc:1034] running on GCE node
W20260806 06:31:55.098363 21418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:55.098362 21414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:55.098767 21408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:55.098990 21408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:55.100114 21408 hybrid_clock.cc:648] HybridClock initialized: now 1785997915100102 us; error 26 us; skew 500 ppm
I20260806 06:31:55.101256 21408 webserver.cc:460] Webserver started at http://127.7.43.67:42921/ using document root <none> and password file <none>
I20260806 06:31:55.101452 21408 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:55.101500 21408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:55.101621 21408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:55.102633 21408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-2/data/instance:
uuid: "9465b8af4868460c8c397b8d3185d2d1"
format_stamp: "Formatted at 2026-08-06 06:31:55 on dist-test-slave-6zbq"
server_key: "086551097f7b2fda3d25cbcdec153ed4"
server_key_iv: "0a6812a33c1ffe2435771a02190a13d2"
server_key_version: "encryptionkey@0"
I20260806 06:31:55.102954 21408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-2/wal/instance:
uuid: "9465b8af4868460c8c397b8d3185d2d1"
format_stamp: "Formatted at 2026-08-06 06:31:55 on dist-test-slave-6zbq"
server_key: "086551097f7b2fda3d25cbcdec153ed4"
server_key_iv: "0a6812a33c1ffe2435771a02190a13d2"
server_key_version: "encryptionkey@0"
I20260806 06:31:55.104094 21408 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:31:55.104687 21429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:55.104847 21408 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:31:55.104924 21408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-2/wal
uuid: "9465b8af4868460c8c397b8d3185d2d1"
format_stamp: "Formatted at 2026-08-06 06:31:55 on dist-test-slave-6zbq"
server_key: "086551097f7b2fda3d25cbcdec153ed4"
server_key_iv: "0a6812a33c1ffe2435771a02190a13d2"
server_key_version: "encryptionkey@0"
I20260806 06:31:55.104981 21408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:55.128345 21408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:55.128556 21408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:55.128641 21408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:55.128801 21408 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:31:55.129046 21408 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:31:55.129076 21408 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:55.129098 21408 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:31:55.129113 21408 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:55.134325 21408 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.67:44685
I20260806 06:31:55.134399 21589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.67:44685 every 8 connection(s)
I20260806 06:31:55.134655 21408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
I20260806 06:31:55.135497 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 21408
I20260806 06:31:55.135573 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-2/wal/instance
I20260806 06:31:55.135766 7341 external_mini_cluster.cc:1428] Setting key 224f7b23555105f0170fe1e7c63f14fe
I20260806 06:31:55.139425 21036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:55.135277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:40373 (local address 127.7.43.126:36375)
0806 06:31:55.135325 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:55.135327 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:55.135558 (+ 231us) server_negotiation.cc:407] Connection header received
0806 06:31:55.136076 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:55.136077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:55.136090 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:55.136135 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:55.136751 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:55.137244 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:55.138140 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:55.138385 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:55.138448 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:55.139159 (+ 711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:55.139161 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:55.139161 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:31:55.139165 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:55.139188 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:31:55.139192 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:55.139238 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:55.139318 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 06:31:55.139348 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:31:55.139590 21593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:55.135265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:36375 (local address 127.7.43.67:40373)
0806 06:31:55.135447 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:55.135483 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:55.135984 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:55.136209 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:55.136213 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:55.136221 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:55.136665 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:55.136669 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:55.137320 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:55.137321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:55.138062 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:55.138065 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:55.138455 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:55.138456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:55.138573 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:55.139026 (+ 453us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:31:55.139026 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:55.139053 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:31:55.139073 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:55.139245 (+ 172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:55.139250 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:55.139253 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:31:55.139313 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 06:31:55.139409 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":94,"threads_started":1}
I20260806 06:31:55.139956 21591 heartbeater.cc:347] Connected to a master server at 127.7.43.126:36375
I20260806 06:31:55.140041 21591 heartbeater.cc:464] Registering TS with master...
I20260806 06:31:55.140205 21591 heartbeater.cc:511] Master 127.7.43.126:36375 requested a full tablet report, sending...
I20260806 06:31:55.140513 20952 ts_manager.cc:194] Registered new tserver with Master: 9465b8af4868460c8c397b8d3185d2d1 (127.7.43.67:44685)
I20260806 06:31:55.140830 20952 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.43.67:40373
I20260806 06:31:55.147230 7341 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 06:31:55.148916 21036 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:55.148444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35572 (local address 127.7.43.126:36375)
0806 06:31:55.148500 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:55.148502 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:55.148609 (+ 107us) server_negotiation.cc:407] Connection header received
0806 06:31:55.148670 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:55.148671 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:55.148682 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:55.148727 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:55.148859 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35572: BlockingRecv error: recv got EOF from 127.0.0.1:35572 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 06:31:55.148978 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 21038
I20260806 06:31:55.155478 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 21221
I20260806 06:31:55.165099 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 21408
I20260806 06:31:55.179237 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 20900
2026-08-06T06:31:55Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (614 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-06T06:31:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:31:55Z Disabled control of system clock
I20260806 06:31:55.208945 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:46823
--webserver_interface=127.7.43.126
--webserver_port=0
--builtin_ntp_servers=127.7.43.84:34665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.43.126:46823
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 06:31:55.286507 21603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:55.286698 21603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:55.286727 21603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:55.287796 21603 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:31:55.287833 21603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:55.287850 21603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:55.287865 21603 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:31:55.287892 21603 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:31:55.289189 21603 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:34665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.43.126:46823
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:46823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.43.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21603
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:55.289414 21603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:55.289649 21603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:55.293938 21621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:55.293942 21616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:55.294117 21603 server_base.cc:1034] running on GCE node
W20260806 06:31:55.293947 21619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:55.294445 21603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:55.294708 21603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:55.295846 21603 hybrid_clock.cc:648] HybridClock initialized: now 1785997915295836 us; error 30 us; skew 500 ppm
I20260806 06:31:55.296890 21603 webserver.cc:460] Webserver started at http://127.7.43.126:45507/ using document root <none> and password file <none>
I20260806 06:31:55.297088 21603 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:55.297133 21603 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:55.297250 21603 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:55.298177 21603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/master-0/data/instance:
uuid: "10c7a51d06814c3780b81efdcb6e5778"
format_stamp: "Formatted at 2026-08-06 06:31:55 on dist-test-slave-6zbq"
server_key: "b05d69906c15dc9b395afdfdb3db0941"
server_key_iv: "562164f4800be3448dd543e243ff6b96"
server_key_version: "encryptionkey@0"
I20260806 06:31:55.298501 21603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/master-0/wal/instance:
uuid: "10c7a51d06814c3780b81efdcb6e5778"
format_stamp: "Formatted at 2026-08-06 06:31:55 on dist-test-slave-6zbq"
server_key: "b05d69906c15dc9b395afdfdb3db0941"
server_key_iv: "562164f4800be3448dd543e243ff6b96"
server_key_version: "encryptionkey@0"
I20260806 06:31:55.299629 21603 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:31:55.300238 21627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:55.300379 21603 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:31:55.300457 21603 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "10c7a51d06814c3780b81efdcb6e5778"
format_stamp: "Formatted at 2026-08-06 06:31:55 on dist-test-slave-6zbq"
server_key: "b05d69906c15dc9b395afdfdb3db0941"
server_key_iv: "562164f4800be3448dd543e243ff6b96"
server_key_version: "encryptionkey@0"
I20260806 06:31:55.300514 21603 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:55.303584 21603 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:55.303808 21603 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:55.303920 21603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:55.307403 21603 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.126:46823
I20260806 06:31:55.307433 21711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.126:46823 every 8 connection(s)
I20260806 06:31:55.307719 21603 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:31:55.308280 21712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:55.310819 21712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10c7a51d06814c3780b81efdcb6e5778: Bootstrap starting.
I20260806 06:31:55.311342 21712 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 10c7a51d06814c3780b81efdcb6e5778: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:55.311594 21712 log.cc:824] T 00000000000000000000000000000000 P 10c7a51d06814c3780b81efdcb6e5778: Log is configured to *not* fsync() on all Append() calls
I20260806 06:31:55.312179 21712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10c7a51d06814c3780b81efdcb6e5778: No bootstrap required, opened a new log
I20260806 06:31:55.312474 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 21603
I20260806 06:31:55.312551 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/master-0/wal/instance
I20260806 06:31:55.312728 7341 external_mini_cluster.cc:1428] Setting key 9a7743ba463ff6b11370d7d799f1236b
I20260806 06:31:55.314065 21712 raft_consensus.cc:348] T 00000000000000000000000000000000 P 10c7a51d06814c3780b81efdcb6e5778 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10c7a51d06814c3780b81efdcb6e5778" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 46823 } }
I20260806 06:31:55.314225 21712 raft_consensus.cc:374] T 00000000000000000000000000000000 P 10c7a51d06814c3780b81efdcb6e5778 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:55.314260 21712 raft_consensus.cc:729] T 00000000000000000000000000000000 P 10c7a51d06814c3780b81efdcb6e5778 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10c7a51d06814c3780b81efdcb6e5778, State: Initialized, Role: FOLLOWER
I20260806 06:31:55.314417 21712 consensus_queue.cc:260] T 00000000000000000000000000000000 P 10c7a51d06814c3780b81efdcb6e5778 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10c7a51d06814c3780b81efdcb6e5778" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 46823 } }
I20260806 06:31:55.314509 21712 raft_consensus.cc:388] T 00000000000000000000000000000000 P 10c7a51d06814c3780b81efdcb6e5778 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:31:55.314548 21712 raft_consensus.cc:482] T 00000000000000000000000000000000 P 10c7a51d06814c3780b81efdcb6e5778 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:31:55.314584 21712 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 10c7a51d06814c3780b81efdcb6e5778 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:55.315193 21712 raft_consensus.cc:504] T 00000000000000000000000000000000 P 10c7a51d06814c3780b81efdcb6e5778 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10c7a51d06814c3780b81efdcb6e5778" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 46823 } }
I20260806 06:31:55.315301 21712 leader_election.cc:302] T 00000000000000000000000000000000 P 10c7a51d06814c3780b81efdcb6e5778 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 10c7a51d06814c3780b81efdcb6e5778; no voters:
I20260806 06:31:55.315320 21717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:55.313170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52880 (local address 127.7.43.126:46823)
0806 06:31:55.313368 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:55.313376 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:55.313408 (+ 32us) server_negotiation.cc:407] Connection header received
0806 06:31:55.314035 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:55.314043 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:55.314092 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:55.314247 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:55.314886 (+ 639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:55.314890 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:55.314890 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:31:55.314894 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:55.314930 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:31:55.314940 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:55.314988 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:55.315062 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 06:31:55.315109 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":84,"threads_started":1}
I20260806 06:31:55.315464 21712 leader_election.cc:288] T 00000000000000000000000000000000 P 10c7a51d06814c3780b81efdcb6e5778 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:31:55.315698 21719 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 10c7a51d06814c3780b81efdcb6e5778 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:31:55.315707 21712 sys_catalog.cc:564] T 00000000000000000000000000000000 P 10c7a51d06814c3780b81efdcb6e5778 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:31:55.315821 21719 raft_consensus.cc:686] T 00000000000000000000000000000000 P 10c7a51d06814c3780b81efdcb6e5778 [term 1 LEADER]: Becoming Leader. State: Replica: 10c7a51d06814c3780b81efdcb6e5778, State: Running, Role: LEADER
I20260806 06:31:55.315898 21719 consensus_queue.cc:237] T 00000000000000000000000000000000 P 10c7a51d06814c3780b81efdcb6e5778 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10c7a51d06814c3780b81efdcb6e5778" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 46823 } }
I20260806 06:31:55.316231 21722 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10c7a51d06814c3780b81efdcb6e5778 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 10c7a51d06814c3780b81efdcb6e5778. Latest consensus state: current_term: 1 leader_uuid: "10c7a51d06814c3780b81efdcb6e5778" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10c7a51d06814c3780b81efdcb6e5778" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 46823 } } }
I20260806 06:31:55.316252 21721 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10c7a51d06814c3780b81efdcb6e5778 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "10c7a51d06814c3780b81efdcb6e5778" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10c7a51d06814c3780b81efdcb6e5778" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 46823 } } }
I20260806 06:31:55.316306 21722 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10c7a51d06814c3780b81efdcb6e5778 [sys.catalog]: This master's current role is: LEADER
I20260806 06:31:55.316334 21721 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10c7a51d06814c3780b81efdcb6e5778 [sys.catalog]: This master's current role is: LEADER
I20260806 06:31:55.317111 21734 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:31:55.317551 21734 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:31:55.318856 21734 catalog_manager.cc:1347] Generated new cluster ID: eeba158ba8714929a5906945144a16a5
I20260806 06:31:55.318897 21734 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:31:55.323164 21734 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:31:55.323292 21734 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:31:55.330631 21734 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 10c7a51d06814c3780b81efdcb6e5778: Generated new TSK 0
I20260806 06:31:55.330771 21734 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:31:55.332228 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.65:0
--local_ip_for_outbound_sockets=127.7.43.65
--webserver_interface=127.7.43.65
--webserver_port=0
--tserver_master_addrs=127.7.43.126:46823
--builtin_ntp_servers=127.7.43.84:34665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 06:31:55.411443 21745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:55.411631 21745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:55.411660 21745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:55.412791 21745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:55.412828 21745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:55.412861 21745 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.65
I20260806 06:31:55.414165 21745 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:34665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.43.65
--webserver_port=0
--tserver_master_addrs=127.7.43.126:46823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21745
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.65
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:55.414397 21745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:55.414641 21745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:55.419324 21751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:55.419446 21755 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:55.419339 21757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:55.419543 21745 server_base.cc:1034] running on GCE node
I20260806 06:31:55.419752 21745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:55.419962 21745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:55.421105 21745 hybrid_clock.cc:648] HybridClock initialized: now 1785997915421083 us; error 30 us; skew 500 ppm
I20260806 06:31:55.422207 21745 webserver.cc:460] Webserver started at http://127.7.43.65:33417/ using document root <none> and password file <none>
I20260806 06:31:55.422453 21745 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:55.422501 21745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:55.422626 21745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:55.423571 21745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-0/data/instance:
uuid: "82af916d845f4dcb97f867d33c5bd5ef"
format_stamp: "Formatted at 2026-08-06 06:31:55 on dist-test-slave-6zbq"
server_key: "d4063a15834339e2f4b5274719fd9fbf"
server_key_iv: "be52303e8694c126f6d3c99ebc28aba8"
server_key_version: "encryptionkey@0"
I20260806 06:31:55.423895 21745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-0/wal/instance:
uuid: "82af916d845f4dcb97f867d33c5bd5ef"
format_stamp: "Formatted at 2026-08-06 06:31:55 on dist-test-slave-6zbq"
server_key: "d4063a15834339e2f4b5274719fd9fbf"
server_key_iv: "be52303e8694c126f6d3c99ebc28aba8"
server_key_version: "encryptionkey@0"
I20260806 06:31:55.425046 21745 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:31:55.425715 21766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:55.425876 21745 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:31:55.425956 21745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-0/wal
uuid: "82af916d845f4dcb97f867d33c5bd5ef"
format_stamp: "Formatted at 2026-08-06 06:31:55 on dist-test-slave-6zbq"
server_key: "d4063a15834339e2f4b5274719fd9fbf"
server_key_iv: "be52303e8694c126f6d3c99ebc28aba8"
server_key_version: "encryptionkey@0"
I20260806 06:31:55.426023 21745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:55.429116 21745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:55.429351 21745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:55.429466 21745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:55.429663 21745 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:31:55.429948 21745 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:31:55.429980 21745 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:55.430006 21745 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:31:55.430028 21745 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:55.436053 21745 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.65:35907
I20260806 06:31:55.436086 21941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.65:35907 every 8 connection(s)
I20260806 06:31:55.436373 21745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
I20260806 06:31:55.437240 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 21745
I20260806 06:31:55.437332 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-0/wal/instance
I20260806 06:31:55.437539 7341 external_mini_cluster.cc:1428] Setting key fe2c103fa96913c8de9f0d6d33d7b595
I20260806 06:31:55.438421 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.66:0
--local_ip_for_outbound_sockets=127.7.43.66
--webserver_interface=127.7.43.66
--webserver_port=0
--tserver_master_addrs=127.7.43.126:46823
--builtin_ntp_servers=127.7.43.84:34665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260806 06:31:55.439301 21717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:55.436967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:41673 (local address 127.7.43.126:46823)
0806 06:31:55.437010 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:55.437012 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:55.437251 (+ 239us) server_negotiation.cc:407] Connection header received
0806 06:31:55.437837 (+ 586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:55.437838 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:55.437846 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:55.437897 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:55.439103 (+ 1206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:55.439104 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:55.439105 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:31:55.439109 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:55.439124 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:31:55.439127 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:55.439173 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:55.439239 (+ 66us) server_negotiation.cc:299] Negotiation successful
0806 06:31:55.439251 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:31:55.439493 21944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:55.436947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:46823 (local address 127.7.43.65:41673)
0806 06:31:55.437132 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:55.437172 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:55.437731 (+ 559us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:55.437959 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:55.437963 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:55.437972 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:55.438951 (+ 979us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:31:55.438951 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:55.438987 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:31:55.439021 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:55.439173 (+ 152us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:55.439177 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:55.439179 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:31:55.439247 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 06:31:55.439310 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":69,"threads_started":1}
I20260806 06:31:55.440032 21942 heartbeater.cc:347] Connected to a master server at 127.7.43.126:46823
I20260806 06:31:55.440112 21942 heartbeater.cc:464] Registering TS with master...
I20260806 06:31:55.440243 21942 heartbeater.cc:511] Master 127.7.43.126:46823 requested a full tablet report, sending...
I20260806 06:31:55.440685 21655 ts_manager.cc:194] Registered new tserver with Master: 82af916d845f4dcb97f867d33c5bd5ef (127.7.43.65:35907)
W20260806 06:31:55.516156 21945 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:55.516330 21945 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:55.516359 21945 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:55.517500 21945 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:55.517537 21945 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:55.517571 21945 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.66
I20260806 06:31:55.518878 21945 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:34665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.43.66
--webserver_port=0
--tserver_master_addrs=127.7.43.126:46823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21945
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.66
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:55.519083 21945 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:55.519330 21945 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:55.523720 21954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:55.523687 21952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:55.523738 21951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:55.523814 21945 server_base.cc:1034] running on GCE node
I20260806 06:31:55.524001 21945 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:55.524217 21945 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:55.525348 21945 hybrid_clock.cc:648] HybridClock initialized: now 1785997915525338 us; error 31 us; skew 500 ppm
I20260806 06:31:55.526607 21945 webserver.cc:460] Webserver started at http://127.7.43.66:38983/ using document root <none> and password file <none>
I20260806 06:31:55.526873 21945 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:55.526937 21945 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:55.527046 21945 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:55.527980 21945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-1/data/instance:
uuid: "3a0d20a9023141f181f6f63f6b6b40d1"
format_stamp: "Formatted at 2026-08-06 06:31:55 on dist-test-slave-6zbq"
server_key: "aea67e6d868d0de4c8d7fb90f1657088"
server_key_iv: "311599a9ad6ec96fbea449aca237a5bf"
server_key_version: "encryptionkey@0"
I20260806 06:31:55.528316 21945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-1/wal/instance:
uuid: "3a0d20a9023141f181f6f63f6b6b40d1"
format_stamp: "Formatted at 2026-08-06 06:31:55 on dist-test-slave-6zbq"
server_key: "aea67e6d868d0de4c8d7fb90f1657088"
server_key_iv: "311599a9ad6ec96fbea449aca237a5bf"
server_key_version: "encryptionkey@0"
I20260806 06:31:55.529457 21945 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:31:55.530046 21962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:55.530198 21945 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:31:55.530275 21945 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-1/wal
uuid: "3a0d20a9023141f181f6f63f6b6b40d1"
format_stamp: "Formatted at 2026-08-06 06:31:55 on dist-test-slave-6zbq"
server_key: "aea67e6d868d0de4c8d7fb90f1657088"
server_key_iv: "311599a9ad6ec96fbea449aca237a5bf"
server_key_version: "encryptionkey@0"
I20260806 06:31:55.530330 21945 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:55.533345 21945 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:55.533555 21945 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:55.533663 21945 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:55.533828 21945 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:31:55.534089 21945 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:31:55.534128 21945 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:55.534153 21945 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:31:55.534176 21945 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:55.539494 21945 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.66:33449
I20260806 06:31:55.539525 22141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.66:33449 every 8 connection(s)
I20260806 06:31:55.539784 21945 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
I20260806 06:31:55.542117 21717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:55.540364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:58133 (local address 127.7.43.126:46823)
0806 06:31:55.540414 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:55.540416 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:55.540602 (+ 186us) server_negotiation.cc:407] Connection header received
0806 06:31:55.540971 (+ 369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:55.540972 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:55.540978 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:55.541024 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:55.541950 (+ 926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:55.541951 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:55.541951 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:31:55.541955 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:55.541969 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:31:55.541973 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:55.542014 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:55.542055 (+ 41us) server_negotiation.cc:299] Negotiation successful
0806 06:31:55.542066 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 06:31:55.542268 22147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:55.540353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:46823 (local address 127.7.43.66:58133)
0806 06:31:55.540500 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:55.540528 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:55.540901 (+ 373us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:55.541068 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:55.541071 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:55.541078 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:55.541834 (+ 756us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:31:55.541834 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:55.541858 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:31:55.541883 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:55.542012 (+ 129us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:55.542015 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:55.542017 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:31:55.542060 (+ 43us) client_negotiation.cc:240] Negotiation successful
0806 06:31:55.542100 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":72,"threads_started":1}
I20260806 06:31:55.542693 22143 heartbeater.cc:347] Connected to a master server at 127.7.43.126:46823
I20260806 06:31:55.542778 22143 heartbeater.cc:464] Registering TS with master...
I20260806 06:31:55.542904 22143 heartbeater.cc:511] Master 127.7.43.126:46823 requested a full tablet report, sending...
I20260806 06:31:55.543241 21655 ts_manager.cc:194] Registered new tserver with Master: 3a0d20a9023141f181f6f63f6b6b40d1 (127.7.43.66:33449)
I20260806 06:31:55.543292 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 21945
I20260806 06:31:55.543390 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-1/wal/instance
I20260806 06:31:55.543582 7341 external_mini_cluster.cc:1428] Setting key 848c5447aca727cee2fdd1badb4f5aa2
I20260806 06:31:55.544294 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.67:0
--local_ip_for_outbound_sockets=127.7.43.67
--webserver_interface=127.7.43.67
--webserver_port=0
--tserver_master_addrs=127.7.43.126:46823
--builtin_ntp_servers=127.7.43.84:34665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 06:31:55.622268 22148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:55.622479 22148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:55.622502 22148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:55.623580 22148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:55.623612 22148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:55.623636 22148 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.67
I20260806 06:31:55.624888 22148 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:34665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.43.67
--webserver_port=0
--tserver_master_addrs=127.7.43.126:46823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22148
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.67
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:55.625046 22148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:55.625252 22148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:55.629623 22160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:55.629667 22148 server_base.cc:1034] running on GCE node
W20260806 06:31:55.629657 22159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:55.629633 22163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:55.629962 22148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:55.630165 22148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:55.631292 22148 hybrid_clock.cc:648] HybridClock initialized: now 1785997915631283 us; error 25 us; skew 500 ppm
I20260806 06:31:55.632356 22148 webserver.cc:460] Webserver started at http://127.7.43.67:42087/ using document root <none> and password file <none>
I20260806 06:31:55.632580 22148 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:55.632627 22148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:55.632735 22148 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:55.633692 22148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-2/data/instance:
uuid: "68068eb1e2764fe1bbc56b785deb7190"
format_stamp: "Formatted at 2026-08-06 06:31:55 on dist-test-slave-6zbq"
server_key: "c8c644fcca604226b6d07eff5e81c601"
server_key_iv: "8e8dc25f69cc465fccb5b2e6d16ae7b0"
server_key_version: "encryptionkey@0"
I20260806 06:31:55.634001 22148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-2/wal/instance:
uuid: "68068eb1e2764fe1bbc56b785deb7190"
format_stamp: "Formatted at 2026-08-06 06:31:55 on dist-test-slave-6zbq"
server_key: "c8c644fcca604226b6d07eff5e81c601"
server_key_iv: "8e8dc25f69cc465fccb5b2e6d16ae7b0"
server_key_version: "encryptionkey@0"
I20260806 06:31:55.635177 22148 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:31:55.635795 22172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:55.635957 22148 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:31:55.636035 22148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-2/wal
uuid: "68068eb1e2764fe1bbc56b785deb7190"
format_stamp: "Formatted at 2026-08-06 06:31:55 on dist-test-slave-6zbq"
server_key: "c8c644fcca604226b6d07eff5e81c601"
server_key_iv: "8e8dc25f69cc465fccb5b2e6d16ae7b0"
server_key_version: "encryptionkey@0"
I20260806 06:31:55.636091 22148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:55.639222 22148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:55.639437 22148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:55.639549 22148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:55.639727 22148 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:31:55.639988 22148 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:31:55.640020 22148 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:55.640045 22148 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:31:55.640069 22148 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:55.645642 22148 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.67:37995
I20260806 06:31:55.645675 22362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.67:37995 every 8 connection(s)
I20260806 06:31:55.645946 22148 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
I20260806 06:31:55.648351 21717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:55.646563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:52707 (local address 127.7.43.126:46823)
0806 06:31:55.646607 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:55.646609 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:55.646797 (+ 188us) server_negotiation.cc:407] Connection header received
0806 06:31:55.647163 (+ 366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:55.647164 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:55.647171 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:55.647221 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:55.648202 (+ 981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:55.648202 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:55.648203 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:31:55.648205 (+ 2us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:55.648214 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:31:55.648217 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:55.648249 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:55.648291 (+ 42us) server_negotiation.cc:299] Negotiation successful
0806 06:31:55.648302 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:31:55.648520 22365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:55.646535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:46823 (local address 127.7.43.67:52707)
0806 06:31:55.646691 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:55.646719 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:55.647093 (+ 374us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:55.647271 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:55.647274 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:55.647281 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:55.648083 (+ 802us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:31:55.648084 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:55.648108 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:31:55.648133 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:55.648247 (+ 114us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:55.648250 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:55.648252 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:31:55.648297 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 06:31:55.648338 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":80,"threads_started":1}
I20260806 06:31:55.648838 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 22148
I20260806 06:31:55.648927 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785997861828271-7341-0/minicluster-data/ts-2/wal/instance
I20260806 06:31:55.648975 22363 heartbeater.cc:347] Connected to a master server at 127.7.43.126:46823
I20260806 06:31:55.649063 22363 heartbeater.cc:464] Registering TS with master...
I20260806 06:31:55.649123 7341 external_mini_cluster.cc:1428] Setting key e2ec6ed6e04a680c9cfa54d574abec2b
I20260806 06:31:55.649205 22363 heartbeater.cc:511] Master 127.7.43.126:46823 requested a full tablet report, sending...
I20260806 06:31:55.649492 21655 ts_manager.cc:194] Registered new tserver with Master: 68068eb1e2764fe1bbc56b785deb7190 (127.7.43.67:37995)
I20260806 06:31:55.649950 7341 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 06:31:55.651546 21717 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:31:55.651161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52882 (local address 127.7.43.126:46823)
0806 06:31:55.651220 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:55.651222 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:55.651298 (+ 76us) server_negotiation.cc:407] Connection header received
0806 06:31:55.651339 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:55.651340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:55.651344 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:55.651383 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:55.651490 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52882: BlockingRecv error: recv got EOF from 127.0.0.1:52882 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 06:31:55.651613 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 21745
I20260806 06:31:55.669353 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 21945
I20260806 06:31:55.677657 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 22148
I20260806 06:31:55.684060 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 21603
2026-08-06T06:31:55Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (505 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:31:55 dist-test-slave-6zbq krb5kdc[22374](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:31:55 dist-test-slave-6zbq krb5kdc[22374](info): set up 2 sockets
Aug 06 06:31:55 dist-test-slave-6zbq krb5kdc[22374](info): commencing operation
krb5kdc: starting...
W20260806 06:31:56.796367 7341 mini_kdc.cc:121] Time spent starting KDC: real 1.089s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Aug 06 06:31:56 dist-test-slave-6zbq krb5kdc[22374](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997916, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:31:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:31:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.43.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
I20260806 06:31:56.898625 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:39793
--webserver_interface=127.7.43.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:42459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.43.126:39793
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:31:56.975387 22392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:56.975561 22392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:56.975581 22392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:56.976640 22392 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:31:56.976673 22392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:56.976696 22392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:56.976713 22392 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:31:56.976724 22392 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:31:56.978084 22392 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:42459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.43.126:39793
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:39793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.43.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22392
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:56.978255 22392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:56.978436 22392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:56.982782 22398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:56.982792 22399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:56.982841 22392 server_base.cc:1034] running on GCE node
W20260806 06:31:56.982807 22402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:56.983260 22392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:56.983520 22392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:56.984650 22392 hybrid_clock.cc:648] HybridClock initialized: now 1785997916984638 us; error 30 us; skew 500 ppm
Aug 06 06:31:56 dist-test-slave-6zbq krb5kdc[22374](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997916, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:56.990881 22392 init.cc:377] Logged in from keytab as kudu/127.7.43.126@KRBTEST.COM (short username kudu)
I20260806 06:31:56.991252 22392 webserver.cc:460] Webserver started at http://127.7.43.126:43639/ using document root <none> and password file <none>
I20260806 06:31:56.991453 22392 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:56.991499 22392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:56.991616 22392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:56.992605 22392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/master-0/data/instance:
uuid: "55e172e6a8fb4d6781df6a8681f1591c"
format_stamp: "Formatted at 2026-08-06 06:31:56 on dist-test-slave-6zbq"
server_key: "82d17b9e190b4476fac45e8d9cee012f"
server_key_iv: "69c1a88d8db89e7c868a2e17b0f557cd"
server_key_version: "encryptionkey@0"
I20260806 06:31:56.992995 22392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/master-0/wal/instance:
uuid: "55e172e6a8fb4d6781df6a8681f1591c"
format_stamp: "Formatted at 2026-08-06 06:31:56 on dist-test-slave-6zbq"
server_key: "82d17b9e190b4476fac45e8d9cee012f"
server_key_iv: "69c1a88d8db89e7c868a2e17b0f557cd"
server_key_version: "encryptionkey@0"
I20260806 06:31:56.994232 22392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 06:31:56.994915 22410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:56.995059 22392 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:56.995136 22392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "55e172e6a8fb4d6781df6a8681f1591c"
format_stamp: "Formatted at 2026-08-06 06:31:56 on dist-test-slave-6zbq"
server_key: "82d17b9e190b4476fac45e8d9cee012f"
server_key_iv: "69c1a88d8db89e7c868a2e17b0f557cd"
server_key_version: "encryptionkey@0"
I20260806 06:31:56.995193 22392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:57.005810 22392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:57.007386 22392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:57.007505 22392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:57.011075 22392 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.126:39793
I20260806 06:31:57.011107 22494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.126:39793 every 8 connection(s)
I20260806 06:31:57.011386 22392 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:31:57.012308 22496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:57.012302 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 22392
I20260806 06:31:57.012395 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/master-0/wal/instance
I20260806 06:31:57.012573 7341 external_mini_cluster.cc:1428] Setting key a8fb51b433216e5cd0ee74a7b6c42b05
I20260806 06:31:57.015165 22496 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55e172e6a8fb4d6781df6a8681f1591c: Bootstrap starting.
I20260806 06:31:57.015779 22496 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 55e172e6a8fb4d6781df6a8681f1591c: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:57.016072 22496 log.cc:824] T 00000000000000000000000000000000 P 55e172e6a8fb4d6781df6a8681f1591c: Log is configured to *not* fsync() on all Append() calls
I20260806 06:31:57.016705 22496 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55e172e6a8fb4d6781df6a8681f1591c: No bootstrap required, opened a new log
Aug 06 06:31:57 dist-test-slave-6zbq krb5kdc[22374](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997916, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:31:57.018743 22496 raft_consensus.cc:348] T 00000000000000000000000000000000 P 55e172e6a8fb4d6781df6a8681f1591c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55e172e6a8fb4d6781df6a8681f1591c" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 39793 } }
I20260806 06:31:57.018908 22496 raft_consensus.cc:374] T 00000000000000000000000000000000 P 55e172e6a8fb4d6781df6a8681f1591c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:57.018950 22496 raft_consensus.cc:729] T 00000000000000000000000000000000 P 55e172e6a8fb4d6781df6a8681f1591c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55e172e6a8fb4d6781df6a8681f1591c, State: Initialized, Role: FOLLOWER
I20260806 06:31:57.019075 22496 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55e172e6a8fb4d6781df6a8681f1591c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55e172e6a8fb4d6781df6a8681f1591c" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 39793 } }
I20260806 06:31:57.019173 22496 raft_consensus.cc:388] T 00000000000000000000000000000000 P 55e172e6a8fb4d6781df6a8681f1591c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:31:57.019243 22496 raft_consensus.cc:482] T 00000000000000000000000000000000 P 55e172e6a8fb4d6781df6a8681f1591c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:31:57.019297 22496 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 55e172e6a8fb4d6781df6a8681f1591c [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:57.019878 22496 raft_consensus.cc:504] T 00000000000000000000000000000000 P 55e172e6a8fb4d6781df6a8681f1591c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55e172e6a8fb4d6781df6a8681f1591c" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 39793 } }
I20260806 06:31:57.019981 22496 leader_election.cc:302] T 00000000000000000000000000000000 P 55e172e6a8fb4d6781df6a8681f1591c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 55e172e6a8fb4d6781df6a8681f1591c; no voters:
I20260806 06:31:57.020143 22496 leader_election.cc:288] T 00000000000000000000000000000000 P 55e172e6a8fb4d6781df6a8681f1591c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:31:57.020202 22501 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 55e172e6a8fb4d6781df6a8681f1591c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:31:57.020373 22501 raft_consensus.cc:686] T 00000000000000000000000000000000 P 55e172e6a8fb4d6781df6a8681f1591c [term 1 LEADER]: Becoming Leader. State: Replica: 55e172e6a8fb4d6781df6a8681f1591c, State: Running, Role: LEADER
I20260806 06:31:57.020411 22496 sys_catalog.cc:564] T 00000000000000000000000000000000 P 55e172e6a8fb4d6781df6a8681f1591c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:31:57.020526 22501 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55e172e6a8fb4d6781df6a8681f1591c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55e172e6a8fb4d6781df6a8681f1591c" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 39793 } }
I20260806 06:31:57.020887 22501 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55e172e6a8fb4d6781df6a8681f1591c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55e172e6a8fb4d6781df6a8681f1591c. Latest consensus state: current_term: 1 leader_uuid: "55e172e6a8fb4d6781df6a8681f1591c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55e172e6a8fb4d6781df6a8681f1591c" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 39793 } } }
I20260806 06:31:57.020941 22504 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55e172e6a8fb4d6781df6a8681f1591c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "55e172e6a8fb4d6781df6a8681f1591c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55e172e6a8fb4d6781df6a8681f1591c" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 39793 } } }
I20260806 06:31:57.021020 22501 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55e172e6a8fb4d6781df6a8681f1591c [sys.catalog]: This master's current role is: LEADER
I20260806 06:31:57.021246 22504 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55e172e6a8fb4d6781df6a8681f1591c [sys.catalog]: This master's current role is: LEADER
I20260806 06:31:57.021277 22512 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:31:57.021867 22512 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:31:57.022842 22499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:57.013040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39960 (local address 127.7.43.126:39793)
0806 06:31:57.013189 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:57.013197 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:57.013222 (+ 25us) server_negotiation.cc:407] Connection header received
0806 06:31:57.013634 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:57.013640 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:57.013680 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:57.013783 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:57.014516 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:57.015416 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:57.016405 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:57.016821 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:57.016916 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:57.019264 (+ 2348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:57.019268 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:57.019268 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:57.019279 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:57.021021 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:57.021426 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:57.021428 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:57.021429 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:57.021523 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:57.021745 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:57.021746 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:57.021746 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:57.022028 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:57.022179 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:57.022535 (+ 356us) server_negotiation.cc:299] Negotiation successful
0806 06:31:57.022660 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":61,"threads_started":1}
I20260806 06:31:57.023592 22512 catalog_manager.cc:1347] Generated new cluster ID: c054416c1a994dda85fcd439fd801b7e
I20260806 06:31:57.023643 22512 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:31:57.038101 22512 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:31:57.038650 22512 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:31:57.052035 22512 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 55e172e6a8fb4d6781df6a8681f1591c: Generated new TSK 0
I20260806 06:31:57.052187 22512 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.43.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab.
Entry for principal kudu/127.7.43.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab.
I20260806 06:31:57.084391 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.65:0
--local_ip_for_outbound_sockets=127.7.43.65
--webserver_interface=127.7.43.65
--webserver_port=0
--tserver_master_addrs=127.7.43.126:39793
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab
--principal=kudu/127.7.43.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:42459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:31:57.167491 22529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:57.167663 22529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:57.167692 22529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:57.168753 22529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:57.168789 22529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:57.168814 22529 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.65
I20260806 06:31:57.170549 22529 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:42459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab
--principal=kudu/127.7.43.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.43.65
--webserver_port=0
--tserver_master_addrs=127.7.43.126:39793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22529
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.65
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:57.170909 22529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:57.171156 22529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:31:57.176082 22529 server_base.cc:1034] running on GCE node
W20260806 06:31:57.176002 22538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:57.176024 22546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:57.176014 22542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:57.176388 22529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:57.176610 22529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:57.177747 22529 hybrid_clock.cc:648] HybridClock initialized: now 1785997917177736 us; error 31 us; skew 500 ppm
Aug 06 06:31:57 dist-test-slave-6zbq krb5kdc[22374](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997917, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:57.179986 22529 init.cc:377] Logged in from keytab as kudu/127.7.43.65@KRBTEST.COM (short username kudu)
I20260806 06:31:57.180363 22529 webserver.cc:460] Webserver started at http://127.7.43.65:41045/ using document root <none> and password file <none>
I20260806 06:31:57.180545 22529 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:57.180590 22529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:57.180711 22529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:57.181636 22529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/ts-0/data/instance:
uuid: "c37f7a263f934e6192c0cf9c743467b6"
format_stamp: "Formatted at 2026-08-06 06:31:57 on dist-test-slave-6zbq"
server_key: "b39985c1aada1c7192b3deca18a0112f"
server_key_iv: "c670257af6a994aa86156a070ce819df"
server_key_version: "encryptionkey@0"
I20260806 06:31:57.181947 22529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/ts-0/wal/instance:
uuid: "c37f7a263f934e6192c0cf9c743467b6"
format_stamp: "Formatted at 2026-08-06 06:31:57 on dist-test-slave-6zbq"
server_key: "b39985c1aada1c7192b3deca18a0112f"
server_key_iv: "c670257af6a994aa86156a070ce819df"
server_key_version: "encryptionkey@0"
I20260806 06:31:57.183151 22529 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:31:57.183760 22555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:57.183900 22529 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:31:57.183977 22529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/ts-0/wal
uuid: "c37f7a263f934e6192c0cf9c743467b6"
format_stamp: "Formatted at 2026-08-06 06:31:57 on dist-test-slave-6zbq"
server_key: "b39985c1aada1c7192b3deca18a0112f"
server_key_iv: "c670257af6a994aa86156a070ce819df"
server_key_version: "encryptionkey@0"
I20260806 06:31:57.184036 22529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:57.213176 22529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:57.215094 22529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:57.215221 22529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:57.215418 22529 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:31:57.215689 22529 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:31:57.215725 22529 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:57.215750 22529 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:31:57.215766 22529 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:57.221170 22529 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.65:39657
I20260806 06:31:57.221230 22724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.65:39657 every 8 connection(s)
I20260806 06:31:57.221491 22529 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
Aug 06 06:31:57 dist-test-slave-6zbq krb5kdc[22374](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997917, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.65@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:31:57.229125 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 22529
I20260806 06:31:57.229204 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/ts-0/wal/instance
I20260806 06:31:57.229374 7341 external_mini_cluster.cc:1428] Setting key 99b3afeb80f0365bb899f4e0328a3b05
I20260806 06:31:57.229529 22499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:57.222143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:57657 (local address 127.7.43.126:39793)
0806 06:31:57.222202 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:57.222205 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:57.222395 (+ 190us) server_negotiation.cc:407] Connection header received
0806 06:31:57.222854 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:57.222855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:57.222868 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:57.222924 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:57.223767 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:57.224251 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:57.225160 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:57.225378 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:57.225432 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:57.227284 (+ 1852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:57.227287 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:57.227287 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:57.227294 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:57.228321 (+ 1027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:57.228657 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:57.228658 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:57.228659 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:57.228712 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:57.228917 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:57.228918 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:57.228919 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:57.229028 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:57.229082 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:57.229359 (+ 277us) server_negotiation.cc:299] Negotiation successful
0806 06:31:57.229441 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 06:31:57.229707 22727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:57.222109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:39793 (local address 127.7.43.65:57657)
0806 06:31:57.222250 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:57.222295 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:57.222765 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:57.222991 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:57.222995 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:57.223502 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:57.223672 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:57.223676 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:57.224330 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:57.224331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:57.225082 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:57.225084 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:57.225445 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:57.225446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:57.225556 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:57.225957 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:57.225964 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:57.227197 (+ 1233us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:57.228389 (+ 1192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:57.228395 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:57.228397 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:57.228593 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:57.228765 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:57.228766 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:57.228766 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:57.228863 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:57.229080 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:57.229086 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:57.229218 (+ 132us) client_negotiation.cc:769] Sending connection context
0806 06:31:57.229328 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 06:31:57.229468 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":75,"threads_started":1}
I20260806 06:31:57.230116 22725 heartbeater.cc:347] Connected to a master server at 127.7.43.126:39793
I20260806 06:31:57.230204 22725 heartbeater.cc:464] Registering TS with master...
I20260806 06:31:57.230393 22725 heartbeater.cc:511] Master 127.7.43.126:39793 requested a full tablet report, sending...
I20260806 06:31:57.230896 22444 ts_manager.cc:194] Registered new tserver with Master: c37f7a263f934e6192c0cf9c743467b6 (127.7.43.65:39657)
I20260806 06:31:57.231492 22444 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.65@KRBTEST.COM'} at 127.7.43.65:57657
WARNING: no policy specified for kudu/127.7.43.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab.
Entry for principal kudu/127.7.43.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab.
I20260806 06:31:57.258538 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.66:0
--local_ip_for_outbound_sockets=127.7.43.66
--webserver_interface=127.7.43.66
--webserver_port=0
--tserver_master_addrs=127.7.43.126:39793
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab
--principal=kudu/127.7.43.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:42459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:31:57.338759 22732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:57.338945 22732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:57.338967 22732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:57.340018 22732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:57.340049 22732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:57.340073 22732 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.66
I20260806 06:31:57.341428 22732 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:42459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab
--principal=kudu/127.7.43.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.43.66
--webserver_port=0
--tserver_master_addrs=127.7.43.126:39793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22732
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.66
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:57.341594 22732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:57.341768 22732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:57.346309 22738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:57.346305 22740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:57.346302 22742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:57.346473 22732 server_base.cc:1034] running on GCE node
I20260806 06:31:57.346660 22732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:57.346899 22732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:57.348031 22732 hybrid_clock.cc:648] HybridClock initialized: now 1785997917348022 us; error 26 us; skew 500 ppm
Aug 06 06:31:57 dist-test-slave-6zbq krb5kdc[22374](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997917, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:57.350373 22732 init.cc:377] Logged in from keytab as kudu/127.7.43.66@KRBTEST.COM (short username kudu)
I20260806 06:31:57.350795 22732 webserver.cc:460] Webserver started at http://127.7.43.66:37039/ using document root <none> and password file <none>
I20260806 06:31:57.350980 22732 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:57.351020 22732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:57.351133 22732 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:57.352098 22732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/ts-1/data/instance:
uuid: "5ed71b7cbbae409bb5eb3929d40f6b10"
format_stamp: "Formatted at 2026-08-06 06:31:57 on dist-test-slave-6zbq"
server_key: "e127c2653db8c2b814dc8aa70434d102"
server_key_iv: "f6977111c76442056a85193fef919faa"
server_key_version: "encryptionkey@0"
I20260806 06:31:57.352416 22732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/ts-1/wal/instance:
uuid: "5ed71b7cbbae409bb5eb3929d40f6b10"
format_stamp: "Formatted at 2026-08-06 06:31:57 on dist-test-slave-6zbq"
server_key: "e127c2653db8c2b814dc8aa70434d102"
server_key_iv: "f6977111c76442056a85193fef919faa"
server_key_version: "encryptionkey@0"
I20260806 06:31:57.353626 22732 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:31:57.354260 22752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:57.354431 22732 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:31:57.354509 22732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/ts-1/wal
uuid: "5ed71b7cbbae409bb5eb3929d40f6b10"
format_stamp: "Formatted at 2026-08-06 06:31:57 on dist-test-slave-6zbq"
server_key: "e127c2653db8c2b814dc8aa70434d102"
server_key_iv: "f6977111c76442056a85193fef919faa"
server_key_version: "encryptionkey@0"
I20260806 06:31:57.354562 22732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:57.365419 22732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:57.367030 22732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:57.367126 22732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:57.367293 22732 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:31:57.367548 22732 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:31:57.367579 22732 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:57.367609 22732 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:31:57.367631 22732 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:57.373101 22732 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.66:37525
I20260806 06:31:57.373157 22917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.66:37525 every 8 connection(s)
I20260806 06:31:57.373410 22732 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
I20260806 06:31:57.373724 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 22732
I20260806 06:31:57.373831 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/ts-1/wal/instance
I20260806 06:31:57.374011 7341 external_mini_cluster.cc:1428] Setting key cb0de84f1792e8923ef6a08d2e1efb28
Aug 06 06:31:57 dist-test-slave-6zbq krb5kdc[22374](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997917, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.66@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:31:57.383031 22499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:57.374079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:35869 (local address 127.7.43.126:39793)
0806 06:31:57.374128 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:57.374130 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:57.374378 (+ 248us) server_negotiation.cc:407] Connection header received
0806 06:31:57.374987 (+ 609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:57.374988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:57.375001 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:57.375063 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:57.375924 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:57.376576 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:57.377854 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:57.378114 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:57.378185 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:57.380434 (+ 2249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:57.380436 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:57.380436 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:57.380443 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:57.381669 (+ 1226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:57.382115 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:57.382116 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:57.382116 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:57.382164 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:57.382452 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:57.382453 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:57.382453 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:57.382554 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:57.382604 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:57.382906 (+ 302us) server_negotiation.cc:299] Negotiation successful
0806 06:31:57.382966 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 06:31:57.383279 22921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:57.374066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:39793 (local address 127.7.43.66:35869)
0806 06:31:57.374236 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:57.374283 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:57.374894 (+ 611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:57.375115 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:57.375118 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:57.375654 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:57.375840 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:57.375844 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:57.376665 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:57.376666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:57.377768 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:57.377771 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:57.378189 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:57.378189 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:57.378292 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:57.378756 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:57.378764 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:57.380343 (+ 1579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:57.381749 (+ 1406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:57.381756 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:57.381759 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:57.382044 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:57.382225 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:57.382226 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:57.382226 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:57.382388 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:57.382613 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:57.382619 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:57.382777 (+ 158us) client_negotiation.cc:769] Sending connection context
0806 06:31:57.382890 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 06:31:57.383053 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":87,"threads_started":1}
WARNING: no policy specified for kudu/127.7.43.67@KRBTEST.COM; defaulting to no policy
I20260806 06:31:57.383750 22918 heartbeater.cc:347] Connected to a master server at 127.7.43.126:39793
I20260806 06:31:57.383843 22918 heartbeater.cc:464] Registering TS with master...
I20260806 06:31:57.384006 22918 heartbeater.cc:511] Master 127.7.43.126:39793 requested a full tablet report, sending...
I20260806 06:31:57.384369 22444 ts_manager.cc:194] Registered new tserver with Master: 5ed71b7cbbae409bb5eb3929d40f6b10 (127.7.43.66:37525)
I20260806 06:31:57.384727 22444 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.66@KRBTEST.COM'} at 127.7.43.66:35869
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab.
Entry for principal kudu/127.7.43.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab.
I20260806 06:31:57.398542 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.67:0
--local_ip_for_outbound_sockets=127.7.43.67
--webserver_interface=127.7.43.67
--webserver_port=0
--tserver_master_addrs=127.7.43.126:39793
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab
--principal=kudu/127.7.43.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:42459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:31:57.477141 22925 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:31:57.477320 22925 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:31:57.477342 22925 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:31:57.478412 22925 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:31:57.478442 22925 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:31:57.478466 22925 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.67
I20260806 06:31:57.479826 22925 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:42459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab
--principal=kudu/127.7.43.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.43.67
--webserver_port=0
--tserver_master_addrs=127.7.43.126:39793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22925
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.67
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:31:57.480000 22925 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:31:57.480162 22925 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:31:57.484687 22935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:31:57.484702 22933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:57.484854 22925 server_base.cc:1034] running on GCE node
W20260806 06:31:57.484696 22931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:31:57.485148 22925 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:31:57.485350 22925 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:31:57.486481 22925 hybrid_clock.cc:648] HybridClock initialized: now 1785997917486467 us; error 28 us; skew 500 ppm
Aug 06 06:31:57 dist-test-slave-6zbq krb5kdc[22374](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997917, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:31:57.488695 22925 init.cc:377] Logged in from keytab as kudu/127.7.43.67@KRBTEST.COM (short username kudu)
I20260806 06:31:57.489069 22925 webserver.cc:460] Webserver started at http://127.7.43.67:40843/ using document root <none> and password file <none>
I20260806 06:31:57.489283 22925 fs_manager.cc:359] Metadata directory not provided
I20260806 06:31:57.489329 22925 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:31:57.489449 22925 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:31:57.490417 22925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/ts-2/data/instance:
uuid: "9fce1171e30449caa882cf50e03e6328"
format_stamp: "Formatted at 2026-08-06 06:31:57 on dist-test-slave-6zbq"
server_key: "c84ffd45eaad60a15dfd8d50f5b4da07"
server_key_iv: "f696eedfb912bed43bc4022772ff1e33"
server_key_version: "encryptionkey@0"
I20260806 06:31:57.490732 22925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/ts-2/wal/instance:
uuid: "9fce1171e30449caa882cf50e03e6328"
format_stamp: "Formatted at 2026-08-06 06:31:57 on dist-test-slave-6zbq"
server_key: "c84ffd45eaad60a15dfd8d50f5b4da07"
server_key_iv: "f696eedfb912bed43bc4022772ff1e33"
server_key_version: "encryptionkey@0"
I20260806 06:31:57.491885 22925 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:31:57.492497 22947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:57.492632 22925 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:31:57.492709 22925 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/ts-2/wal
uuid: "9fce1171e30449caa882cf50e03e6328"
format_stamp: "Formatted at 2026-08-06 06:31:57 on dist-test-slave-6zbq"
server_key: "c84ffd45eaad60a15dfd8d50f5b4da07"
server_key_iv: "f696eedfb912bed43bc4022772ff1e33"
server_key_version: "encryptionkey@0"
I20260806 06:31:57.492774 22925 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:31:57.511152 22925 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:31:57.512763 22925 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:31:57.512861 22925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:31:57.513013 22925 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:31:57.513283 22925 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:31:57.513314 22925 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:57.513340 22925 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:31:57.513365 22925 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:31:57.518865 22925 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.67:34035
I20260806 06:31:57.518920 23119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.67:34035 every 8 connection(s)
I20260806 06:31:57.519176 22925 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
I20260806 06:31:57.523003 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 22925
I20260806 06:31:57.523083 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/ts-2/wal/instance
I20260806 06:31:57.523283 7341 external_mini_cluster.cc:1428] Setting key e265d76fc0874a8b77d7a77adf9ef02d
Aug 06 06:31:57 dist-test-slave-6zbq krb5kdc[22374](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997917, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.67@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:31:57.527597 22499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:57.519804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:41525 (local address 127.7.43.126:39793)
0806 06:31:57.519873 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:57.519875 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:57.520023 (+ 148us) server_negotiation.cc:407] Connection header received
0806 06:31:57.520563 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:57.520564 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:57.520574 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:57.520619 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:57.521535 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:57.522016 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:57.523170 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:57.523442 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:57.523507 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:57.525461 (+ 1954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:57.525462 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:57.525463 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:57.525469 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:57.526530 (+ 1061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:57.526941 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:57.526942 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:57.526942 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:57.526991 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:57.527164 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:57.527165 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:57.527165 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:57.527253 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:57.527296 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:57.527482 (+ 186us) server_negotiation.cc:299] Negotiation successful
0806 06:31:57.527538 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 06:31:57.527733 23123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:57.519777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:39793 (local address 127.7.43.67:41525)
0806 06:31:57.519908 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:57.519941 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:57.520484 (+ 543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:57.520676 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:57.520679 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:57.521237 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:31:57.521463 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:57.521469 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:57.522080 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:57.522081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:57.523057 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:57.523060 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:57.523540 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:57.523541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:57.523652 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:57.524034 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:31:57.524043 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:31:57.525383 (+ 1340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:31:57.526609 (+ 1226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:57.526615 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:57.526617 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:57.526876 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:57.527035 (+ 159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:31:57.527036 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:31:57.527036 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:31:57.527123 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:31:57.527293 (+ 170us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:31:57.527296 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:31:57.527394 (+ 98us) client_negotiation.cc:769] Sending connection context
0806 06:31:57.527465 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 06:31:57.527570 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":71,"threads_started":1}
I20260806 06:31:57.528153 23120 heartbeater.cc:347] Connected to a master server at 127.7.43.126:39793
I20260806 06:31:57.528237 23120 heartbeater.cc:464] Registering TS with master...
I20260806 06:31:57.528393 23120 heartbeater.cc:511] Master 127.7.43.126:39793 requested a full tablet report, sending...
I20260806 06:31:57.528756 22444 ts_manager.cc:194] Registered new tserver with Master: 9fce1171e30449caa882cf50e03e6328 (127.7.43.67:34035)
I20260806 06:31:57.529079 22444 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.67@KRBTEST.COM'} at 127.7.43.67:41525
I20260806 06:31:57.534720 7341 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:31:57.540639 22499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:57.535870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39976 (local address 127.7.43.126:39793)
0806 06:31:57.535911 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:57.535912 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:57.535986 (+ 74us) server_negotiation.cc:407] Connection header received
0806 06:31:57.536026 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:57.536027 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:57.536035 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:57.536078 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:31:57.536521 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:57.536960 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:57.537594 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:57.537770 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:57.537816 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:57.538844 (+ 1028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:31:57.538845 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:31:57.538845 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:31:57.538852 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:31:57.539855 (+ 1003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:57.540116 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:57.540116 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:57.540117 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:57.540160 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:31:57.540282 (+ 122us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:31:57.540282 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:31:57.540282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:31:57.540388 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:31:57.540453 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:57.540521 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 06:31:57.540572 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 06:31:57.542070 22444 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39976:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:31:57.543617 22444 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:31:57.549386 23144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:57.545570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:39657 (local address 127.0.0.1:34330)
0806 06:31:57.545909 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:57.545920 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:57.545976 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:57.546245 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:57.546246 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:57.546247 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:31:57.546307 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:57.546309 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:57.547253 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:57.547254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:57.548406 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:57.548409 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:57.549098 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:57.549100 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:57.549188 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:57.549202 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 06:31:57.549261 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 06:31:57.549310 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":63,"threads_started":1}
I20260806 06:31:57.549394 23137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:57.545686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34330 (local address 127.7.43.65:39657)
0806 06:31:57.545943 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:57.545947 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:57.545958 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:31:57.546067 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:57.546069 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:57.546084 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:57.546150 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:31:57.546432 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:57.547162 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:57.548496 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:57.549030 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:57.549090 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:57.549155 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:57.549286 (+ 131us) server_negotiation.cc:299] Negotiation successful
0806 06:31:57.549322 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":86,"threads_started":1}
I20260806 06:31:57.549620 23136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:57.545810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.67:34035 (local address 127.0.0.1:52018)
0806 06:31:57.546158 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:57.546166 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:57.546213 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:57.546552 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:57.546552 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:57.546554 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:31:57.546595 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:57.546597 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:57.547726 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:57.547727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:57.548502 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:57.548504 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:57.549409 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:57.549410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:57.549468 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:57.549472 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:31:57.549515 (+ 43us) client_negotiation.cc:240] Negotiation successful
0806 06:31:57.549552 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":58,"threads_started":1}
I20260806 06:31:57.549722 23134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:57.545566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.66:37525 (local address 127.0.0.1:45402)
0806 06:31:57.545861 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:57.545900 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:57.545972 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:57.546222 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:57.546225 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:57.546227 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:31:57.546289 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:57.546291 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:57.547586 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:57.547587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:57.548635 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:57.548637 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:57.549530 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:57.549530 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:57.549593 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:57.549596 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:31:57.549636 (+ 40us) client_negotiation.cc:240] Negotiation successful
0806 06:31:57.549669 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":68,"threads_started":1}
I20260806 06:31:57.549831 23146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:57.546017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52018 (local address 127.7.43.67:34035)
0806 06:31:57.546155 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:57.546159 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:57.546334 (+ 175us) server_negotiation.cc:407] Connection header received
0806 06:31:57.546394 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:57.546396 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:57.546410 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:57.546476 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:31:57.546665 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:57.547647 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:57.548571 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:57.549234 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:57.549311 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:57.549371 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:57.549723 (+ 352us) server_negotiation.cc:299] Negotiation successful
0806 06:31:57.549761 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":73,"threads_started":1}
I20260806 06:31:57.549978 23142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:57.545680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45402 (local address 127.7.43.66:37525)
0806 06:31:57.545852 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:57.545857 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:57.546061 (+ 204us) server_negotiation.cc:407] Connection header received
0806 06:31:57.546101 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:57.546105 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:57.546141 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:57.546211 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:31:57.546391 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:57.547486 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:57.548706 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:57.549392 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:57.549450 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:57.549499 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:57.549863 (+ 364us) server_negotiation.cc:299] Negotiation successful
0806 06:31:57.549903 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":94,"threads_started":1}
I20260806 06:31:57.550235 22624 tablet_service.cc:1467] Processing CreateTablet for tablet 07d54b594c6a4ff39a8ea85801c64c9d (DEFAULT_TABLE table=test-table [id=d80f4e4076a740329f85ad6828409bb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:31:57.550328 23016 tablet_service.cc:1467] Processing CreateTablet for tablet 07d54b594c6a4ff39a8ea85801c64c9d (DEFAULT_TABLE table=test-table [id=d80f4e4076a740329f85ad6828409bb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:31:57.550568 22624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07d54b594c6a4ff39a8ea85801c64c9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:57.550680 23016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07d54b594c6a4ff39a8ea85801c64c9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:57.553035 23149 tablet_bootstrap.cc:492] T 07d54b594c6a4ff39a8ea85801c64c9d P c37f7a263f934e6192c0cf9c743467b6: Bootstrap starting.
I20260806 06:31:57.553138 23147 tablet_bootstrap.cc:492] T 07d54b594c6a4ff39a8ea85801c64c9d P 9fce1171e30449caa882cf50e03e6328: Bootstrap starting.
I20260806 06:31:57.553524 22817 tablet_service.cc:1467] Processing CreateTablet for tablet 07d54b594c6a4ff39a8ea85801c64c9d (DEFAULT_TABLE table=test-table [id=d80f4e4076a740329f85ad6828409bb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:31:57.553777 23147 tablet_bootstrap.cc:654] T 07d54b594c6a4ff39a8ea85801c64c9d P 9fce1171e30449caa882cf50e03e6328: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:57.553800 23149 tablet_bootstrap.cc:654] T 07d54b594c6a4ff39a8ea85801c64c9d P c37f7a263f934e6192c0cf9c743467b6: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:57.553822 22817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07d54b594c6a4ff39a8ea85801c64c9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:31:57.554219 23149 log.cc:824] T 07d54b594c6a4ff39a8ea85801c64c9d P c37f7a263f934e6192c0cf9c743467b6: Log is configured to *not* fsync() on all Append() calls
I20260806 06:31:57.554369 23147 log.cc:824] T 07d54b594c6a4ff39a8ea85801c64c9d P 9fce1171e30449caa882cf50e03e6328: Log is configured to *not* fsync() on all Append() calls
I20260806 06:31:57.555141 23147 tablet_bootstrap.cc:492] T 07d54b594c6a4ff39a8ea85801c64c9d P 9fce1171e30449caa882cf50e03e6328: No bootstrap required, opened a new log
I20260806 06:31:57.555214 23147 ts_tablet_manager.cc:1397] T 07d54b594c6a4ff39a8ea85801c64c9d P 9fce1171e30449caa882cf50e03e6328: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:31:57.556921 23153 tablet_bootstrap.cc:492] T 07d54b594c6a4ff39a8ea85801c64c9d P 5ed71b7cbbae409bb5eb3929d40f6b10: Bootstrap starting.
I20260806 06:31:57.557106 23147 raft_consensus.cc:348] T 07d54b594c6a4ff39a8ea85801c64c9d P 9fce1171e30449caa882cf50e03e6328 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c37f7a263f934e6192c0cf9c743467b6" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 39657 } } peers { permanent_uuid: "5ed71b7cbbae409bb5eb3929d40f6b10" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37525 } } peers { permanent_uuid: "9fce1171e30449caa882cf50e03e6328" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 34035 } }
I20260806 06:31:57.557231 23147 raft_consensus.cc:374] T 07d54b594c6a4ff39a8ea85801c64c9d P 9fce1171e30449caa882cf50e03e6328 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:57.557262 23147 raft_consensus.cc:729] T 07d54b594c6a4ff39a8ea85801c64c9d P 9fce1171e30449caa882cf50e03e6328 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fce1171e30449caa882cf50e03e6328, State: Initialized, Role: FOLLOWER
I20260806 06:31:57.557370 23147 consensus_queue.cc:260] T 07d54b594c6a4ff39a8ea85801c64c9d P 9fce1171e30449caa882cf50e03e6328 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c37f7a263f934e6192c0cf9c743467b6" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 39657 } } peers { permanent_uuid: "5ed71b7cbbae409bb5eb3929d40f6b10" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37525 } } peers { permanent_uuid: "9fce1171e30449caa882cf50e03e6328" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 34035 } }
I20260806 06:31:57.557507 23153 tablet_bootstrap.cc:654] T 07d54b594c6a4ff39a8ea85801c64c9d P 5ed71b7cbbae409bb5eb3929d40f6b10: Neither blocks nor log segments found. Creating new log.
I20260806 06:31:57.557638 23147 ts_tablet_manager.cc:1428] T 07d54b594c6a4ff39a8ea85801c64c9d P 9fce1171e30449caa882cf50e03e6328: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 06:31:57.557780 23120 heartbeater.cc:503] Master 127.7.43.126:39793 was elected leader, sending a full tablet report...
I20260806 06:31:57.558048 23153 log.cc:824] T 07d54b594c6a4ff39a8ea85801c64c9d P 5ed71b7cbbae409bb5eb3929d40f6b10: Log is configured to *not* fsync() on all Append() calls
I20260806 06:31:57.558813 23153 tablet_bootstrap.cc:492] T 07d54b594c6a4ff39a8ea85801c64c9d P 5ed71b7cbbae409bb5eb3929d40f6b10: No bootstrap required, opened a new log
I20260806 06:31:57.558874 23153 ts_tablet_manager.cc:1397] T 07d54b594c6a4ff39a8ea85801c64c9d P 5ed71b7cbbae409bb5eb3929d40f6b10: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:31:57.560173 23153 raft_consensus.cc:348] T 07d54b594c6a4ff39a8ea85801c64c9d P 5ed71b7cbbae409bb5eb3929d40f6b10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c37f7a263f934e6192c0cf9c743467b6" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 39657 } } peers { permanent_uuid: "5ed71b7cbbae409bb5eb3929d40f6b10" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37525 } } peers { permanent_uuid: "9fce1171e30449caa882cf50e03e6328" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 34035 } }
I20260806 06:31:57.560300 23153 raft_consensus.cc:374] T 07d54b594c6a4ff39a8ea85801c64c9d P 5ed71b7cbbae409bb5eb3929d40f6b10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:57.560333 23153 raft_consensus.cc:729] T 07d54b594c6a4ff39a8ea85801c64c9d P 5ed71b7cbbae409bb5eb3929d40f6b10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ed71b7cbbae409bb5eb3929d40f6b10, State: Initialized, Role: FOLLOWER
I20260806 06:31:57.560436 23153 consensus_queue.cc:260] T 07d54b594c6a4ff39a8ea85801c64c9d P 5ed71b7cbbae409bb5eb3929d40f6b10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c37f7a263f934e6192c0cf9c743467b6" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 39657 } } peers { permanent_uuid: "5ed71b7cbbae409bb5eb3929d40f6b10" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37525 } } peers { permanent_uuid: "9fce1171e30449caa882cf50e03e6328" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 34035 } }
I20260806 06:31:57.560653 23153 ts_tablet_manager.cc:1428] T 07d54b594c6a4ff39a8ea85801c64c9d P 5ed71b7cbbae409bb5eb3929d40f6b10: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:31:57.560694 22918 heartbeater.cc:503] Master 127.7.43.126:39793 was elected leader, sending a full tablet report...
I20260806 06:31:57.561035 23149 tablet_bootstrap.cc:492] T 07d54b594c6a4ff39a8ea85801c64c9d P c37f7a263f934e6192c0cf9c743467b6: No bootstrap required, opened a new log
I20260806 06:31:57.561125 23149 ts_tablet_manager.cc:1397] T 07d54b594c6a4ff39a8ea85801c64c9d P c37f7a263f934e6192c0cf9c743467b6: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20260806 06:31:57.562630 23149 raft_consensus.cc:348] T 07d54b594c6a4ff39a8ea85801c64c9d P c37f7a263f934e6192c0cf9c743467b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c37f7a263f934e6192c0cf9c743467b6" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 39657 } } peers { permanent_uuid: "5ed71b7cbbae409bb5eb3929d40f6b10" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37525 } } peers { permanent_uuid: "9fce1171e30449caa882cf50e03e6328" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 34035 } }
I20260806 06:31:57.562753 23149 raft_consensus.cc:374] T 07d54b594c6a4ff39a8ea85801c64c9d P c37f7a263f934e6192c0cf9c743467b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:31:57.562773 23149 raft_consensus.cc:729] T 07d54b594c6a4ff39a8ea85801c64c9d P c37f7a263f934e6192c0cf9c743467b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c37f7a263f934e6192c0cf9c743467b6, State: Initialized, Role: FOLLOWER
I20260806 06:31:57.562844 23149 consensus_queue.cc:260] T 07d54b594c6a4ff39a8ea85801c64c9d P c37f7a263f934e6192c0cf9c743467b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c37f7a263f934e6192c0cf9c743467b6" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 39657 } } peers { permanent_uuid: "5ed71b7cbbae409bb5eb3929d40f6b10" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37525 } } peers { permanent_uuid: "9fce1171e30449caa882cf50e03e6328" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 34035 } }
I20260806 06:31:57.563020 23149 ts_tablet_manager.cc:1428] T 07d54b594c6a4ff39a8ea85801c64c9d P c37f7a263f934e6192c0cf9c743467b6: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:31:57.563087 22725 heartbeater.cc:503] Master 127.7.43.126:39793 was elected leader, sending a full tablet report...
W20260806 06:31:57.624050 22919 tablet.cc:2369] T 07d54b594c6a4ff39a8ea85801c64c9d P 5ed71b7cbbae409bb5eb3929d40f6b10: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 06:31:57.722118 22726 tablet.cc:2369] T 07d54b594c6a4ff39a8ea85801c64c9d P c37f7a263f934e6192c0cf9c743467b6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:31:57.723945 23157 raft_consensus.cc:482] T 07d54b594c6a4ff39a8ea85801c64c9d P 5ed71b7cbbae409bb5eb3929d40f6b10 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:31:57.724041 23157 raft_consensus.cc:504] T 07d54b594c6a4ff39a8ea85801c64c9d P 5ed71b7cbbae409bb5eb3929d40f6b10 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c37f7a263f934e6192c0cf9c743467b6" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 39657 } } peers { permanent_uuid: "5ed71b7cbbae409bb5eb3929d40f6b10" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37525 } } peers { permanent_uuid: "9fce1171e30449caa882cf50e03e6328" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 34035 } }
I20260806 06:31:57.724407 23157 leader_election.cc:288] T 07d54b594c6a4ff39a8ea85801c64c9d P 5ed71b7cbbae409bb5eb3929d40f6b10 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c37f7a263f934e6192c0cf9c743467b6 (127.7.43.65:39657), 9fce1171e30449caa882cf50e03e6328 (127.7.43.67:34035)
I20260806 06:31:57.727041 23137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:57.724718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:60339 (local address 127.7.43.65:39657)
0806 06:31:57.724766 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:57.724768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:57.724777 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:31:57.724801 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:57.724802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:57.724815 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:57.724863 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:31:57.724992 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:57.725459 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:57.726334 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:57.726722 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:57.726784 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:57.726796 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:57.726946 (+ 150us) server_negotiation.cc:299] Negotiation successful
0806 06:31:57.726981 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 06:31:57.727041 22921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:57.724563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:39657 (local address 127.7.43.66:60339)
0806 06:31:57.724610 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:57.724621 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:57.724662 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:57.724870 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:57.724871 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:57.724873 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:31:57.724936 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:57.724938 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:57.725531 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:57.725532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:57.726249 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:57.726251 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:57.726800 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:57.726801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:57.726886 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:57.726889 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:31:57.726935 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 06:31:57.726975 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260806 06:31:57.727293 23146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:57.724757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:58985 (local address 127.7.43.67:34035)
0806 06:31:57.724816 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:57.724818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:57.724922 (+ 104us) server_negotiation.cc:407] Connection header received
0806 06:31:57.724981 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:57.724982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:57.724993 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:57.725041 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:31:57.725188 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:57.725642 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:57.726571 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:57.726937 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:57.726999 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:57.727009 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:57.727186 (+ 177us) server_negotiation.cc:299] Negotiation successful
0806 06:31:57.727228 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 06:31:57.727346 22652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07d54b594c6a4ff39a8ea85801c64c9d" candidate_uuid: "5ed71b7cbbae409bb5eb3929d40f6b10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c37f7a263f934e6192c0cf9c743467b6" is_pre_election: true
I20260806 06:31:57.727502 22652 raft_consensus.cc:2454] T 07d54b594c6a4ff39a8ea85801c64c9d P c37f7a263f934e6192c0cf9c743467b6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ed71b7cbbae409bb5eb3929d40f6b10 in term 0.
I20260806 06:31:57.727523 23159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:57.724661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.67:34035 (local address 127.7.43.66:58985)
0806 06:31:57.724855 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:57.724864 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:31:57.724926 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:31:57.725051 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:31:57.725052 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:31:57.725053 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:31:57.725131 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:57.725132 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:57.725714 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:57.725715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:57.726484 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:31:57.726487 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:31:57.727013 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:31:57.727014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:31:57.727094 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:57.727098 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:31:57.727407 (+ 309us) client_negotiation.cc:240] Negotiation successful
0806 06:31:57.727449 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":113,"threads_started":1}
I20260806 06:31:57.727737 22754 leader_election.cc:302] T 07d54b594c6a4ff39a8ea85801c64c9d P 5ed71b7cbbae409bb5eb3929d40f6b10 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ed71b7cbbae409bb5eb3929d40f6b10, c37f7a263f934e6192c0cf9c743467b6; no voters:
I20260806 06:31:57.727866 23157 raft_consensus.cc:2781] T 07d54b594c6a4ff39a8ea85801c64c9d P 5ed71b7cbbae409bb5eb3929d40f6b10 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:31:57.727921 23157 raft_consensus.cc:482] T 07d54b594c6a4ff39a8ea85801c64c9d P 5ed71b7cbbae409bb5eb3929d40f6b10 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:31:57.727869 23051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07d54b594c6a4ff39a8ea85801c64c9d" candidate_uuid: "5ed71b7cbbae409bb5eb3929d40f6b10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fce1171e30449caa882cf50e03e6328" is_pre_election: true
I20260806 06:31:57.727944 23157 raft_consensus.cc:3037] T 07d54b594c6a4ff39a8ea85801c64c9d P 5ed71b7cbbae409bb5eb3929d40f6b10 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:57.727999 23051 raft_consensus.cc:2454] T 07d54b594c6a4ff39a8ea85801c64c9d P 9fce1171e30449caa882cf50e03e6328 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ed71b7cbbae409bb5eb3929d40f6b10 in term 0.
I20260806 06:31:57.728659 23157 raft_consensus.cc:504] T 07d54b594c6a4ff39a8ea85801c64c9d P 5ed71b7cbbae409bb5eb3929d40f6b10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c37f7a263f934e6192c0cf9c743467b6" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 39657 } } peers { permanent_uuid: "5ed71b7cbbae409bb5eb3929d40f6b10" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37525 } } peers { permanent_uuid: "9fce1171e30449caa882cf50e03e6328" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 34035 } }
I20260806 06:31:57.728765 23157 leader_election.cc:288] T 07d54b594c6a4ff39a8ea85801c64c9d P 5ed71b7cbbae409bb5eb3929d40f6b10 [CANDIDATE]: Term 1 election: Requested vote from peers c37f7a263f934e6192c0cf9c743467b6 (127.7.43.65:39657), 9fce1171e30449caa882cf50e03e6328 (127.7.43.67:34035)
I20260806 06:31:57.728951 22652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07d54b594c6a4ff39a8ea85801c64c9d" candidate_uuid: "5ed71b7cbbae409bb5eb3929d40f6b10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c37f7a263f934e6192c0cf9c743467b6"
I20260806 06:31:57.728981 23051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07d54b594c6a4ff39a8ea85801c64c9d" candidate_uuid: "5ed71b7cbbae409bb5eb3929d40f6b10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fce1171e30449caa882cf50e03e6328"
I20260806 06:31:57.729045 22652 raft_consensus.cc:3037] T 07d54b594c6a4ff39a8ea85801c64c9d P c37f7a263f934e6192c0cf9c743467b6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:57.729045 23051 raft_consensus.cc:3037] T 07d54b594c6a4ff39a8ea85801c64c9d P 9fce1171e30449caa882cf50e03e6328 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:31:57.729890 23051 raft_consensus.cc:2454] T 07d54b594c6a4ff39a8ea85801c64c9d P 9fce1171e30449caa882cf50e03e6328 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ed71b7cbbae409bb5eb3929d40f6b10 in term 1.
I20260806 06:31:57.729890 22652 raft_consensus.cc:2454] T 07d54b594c6a4ff39a8ea85801c64c9d P c37f7a263f934e6192c0cf9c743467b6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ed71b7cbbae409bb5eb3929d40f6b10 in term 1.
I20260806 06:31:57.730055 22754 leader_election.cc:302] T 07d54b594c6a4ff39a8ea85801c64c9d P 5ed71b7cbbae409bb5eb3929d40f6b10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ed71b7cbbae409bb5eb3929d40f6b10, 9fce1171e30449caa882cf50e03e6328; no voters:
I20260806 06:31:57.730131 23157 raft_consensus.cc:2781] T 07d54b594c6a4ff39a8ea85801c64c9d P 5ed71b7cbbae409bb5eb3929d40f6b10 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:31:57.730275 23157 raft_consensus.cc:686] T 07d54b594c6a4ff39a8ea85801c64c9d P 5ed71b7cbbae409bb5eb3929d40f6b10 [term 1 LEADER]: Becoming Leader. State: Replica: 5ed71b7cbbae409bb5eb3929d40f6b10, State: Running, Role: LEADER
I20260806 06:31:57.730386 23157 consensus_queue.cc:237] T 07d54b594c6a4ff39a8ea85801c64c9d P 5ed71b7cbbae409bb5eb3929d40f6b10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c37f7a263f934e6192c0cf9c743467b6" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 39657 } } peers { permanent_uuid: "5ed71b7cbbae409bb5eb3929d40f6b10" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37525 } } peers { permanent_uuid: "9fce1171e30449caa882cf50e03e6328" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 34035 } }
I20260806 06:31:57.731091 22443 catalog_manager.cc:5495] T 07d54b594c6a4ff39a8ea85801c64c9d P 5ed71b7cbbae409bb5eb3929d40f6b10 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ed71b7cbbae409bb5eb3929d40f6b10 (127.7.43.66). New cstate: current_term: 1 leader_uuid: "5ed71b7cbbae409bb5eb3929d40f6b10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c37f7a263f934e6192c0cf9c743467b6" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 39657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ed71b7cbbae409bb5eb3929d40f6b10" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 37525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9fce1171e30449caa882cf50e03e6328" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 34035 } health_report { overall_health: UNKNOWN } } }
W20260806 06:31:57.769668 23121 tablet.cc:2369] T 07d54b594c6a4ff39a8ea85801c64c9d P 9fce1171e30449caa882cf50e03e6328: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:31:57.772354 23142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:57.770092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45412 (local address 127.7.43.66:37525)
0806 06:31:57.770138 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 06:31:57.770140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:31:57.770153 (+ 13us) server_negotiation.cc:407] Connection header received
0806 06:31:57.770239 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:31:57.770240 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:31:57.770259 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:31:57.770314 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:31:57.770472 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:57.770927 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:57.771646 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:31:57.771838 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:31:57.771890 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:31:57.772044 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:31:57.772143 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:31:57.772213 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 06:31:57.772252 (+ 39us) server_negotiation.cc:299] Negotiation successful
0806 06:31:57.772270 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 06:31:57.773967 23051 raft_consensus.cc:1261] T 07d54b594c6a4ff39a8ea85801c64c9d P 9fce1171e30449caa882cf50e03e6328 [term 1 FOLLOWER]: Refusing update from remote peer 5ed71b7cbbae409bb5eb3929d40f6b10: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:31:57.773967 22652 raft_consensus.cc:1261] T 07d54b594c6a4ff39a8ea85801c64c9d P c37f7a263f934e6192c0cf9c743467b6 [term 1 FOLLOWER]: Refusing update from remote peer 5ed71b7cbbae409bb5eb3929d40f6b10: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:31:57.774291 23157 consensus_queue.cc:1035] T 07d54b594c6a4ff39a8ea85801c64c9d P 5ed71b7cbbae409bb5eb3929d40f6b10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c37f7a263f934e6192c0cf9c743467b6" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 39657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:31:57.774453 23161 consensus_queue.cc:1035] T 07d54b594c6a4ff39a8ea85801c64c9d P 5ed71b7cbbae409bb5eb3929d40f6b10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9fce1171e30449caa882cf50e03e6328" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 34035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:31:57.776736 23166 mvcc.cc:205] Tried to move back new op lower bound from 7315447471199768576 to 7315447471024160768. Current Snapshot: MvccSnapshot[applied={T|T < 7315447471199768576}]
I20260806 06:31:57.778537 22443 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39976:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:31:57.778626 22443 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39976:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:31:57.779335 22443 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 55e172e6a8fb4d6781df6a8681f1591c: Sending DeleteTablet for 3 replicas of tablet 07d54b594c6a4ff39a8ea85801c64c9d
I20260806 06:31:57.779619 22817 tablet_service.cc:1514] Processing DeleteTablet for tablet 07d54b594c6a4ff39a8ea85801c64c9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:31:57 UTC) from {username='kudu', principal='kudu/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:45402
I20260806 06:31:57.779670 23016 tablet_service.cc:1514] Processing DeleteTablet for tablet 07d54b594c6a4ff39a8ea85801c64c9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:31:57 UTC) from {username='kudu', principal='kudu/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:52018
I20260806 06:31:57.779948 22624 tablet_service.cc:1514] Processing DeleteTablet for tablet 07d54b594c6a4ff39a8ea85801c64c9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:31:57 UTC) from {username='kudu', principal='kudu/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:34330
I20260806 06:31:57.780051 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 22529
I20260806 06:31:57.785544 23177 tablet_replica.cc:330] T 07d54b594c6a4ff39a8ea85801c64c9d P 5ed71b7cbbae409bb5eb3929d40f6b10: stopping tablet replica
I20260806 06:31:57.785638 23177 raft_consensus.cc:2229] T 07d54b594c6a4ff39a8ea85801c64c9d P 5ed71b7cbbae409bb5eb3929d40f6b10 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:31:57.785724 23177 raft_consensus.cc:2258] T 07d54b594c6a4ff39a8ea85801c64c9d P 5ed71b7cbbae409bb5eb3929d40f6b10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:31:57.785873 23178 tablet_replica.cc:330] T 07d54b594c6a4ff39a8ea85801c64c9d P 9fce1171e30449caa882cf50e03e6328: stopping tablet replica
I20260806 06:31:57.785961 23178 raft_consensus.cc:2229] T 07d54b594c6a4ff39a8ea85801c64c9d P 9fce1171e30449caa882cf50e03e6328 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:31:57.786036 23178 raft_consensus.cc:2258] T 07d54b594c6a4ff39a8ea85801c64c9d P 9fce1171e30449caa882cf50e03e6328 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:31:57.787122 23178 ts_tablet_manager.cc:1905] T 07d54b594c6a4ff39a8ea85801c64c9d P 9fce1171e30449caa882cf50e03e6328: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:31:57.788791 23178 ts_tablet_manager.cc:1918] T 07d54b594c6a4ff39a8ea85801c64c9d P 9fce1171e30449caa882cf50e03e6328: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:31:57.788872 23178 log.cc:1196] T 07d54b594c6a4ff39a8ea85801c64c9d P 9fce1171e30449caa882cf50e03e6328: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/ts-2/wal/wals/07d54b594c6a4ff39a8ea85801c64c9d
I20260806 06:31:57.789132 23178 ts_tablet_manager.cc:1939] T 07d54b594c6a4ff39a8ea85801c64c9d P 9fce1171e30449caa882cf50e03e6328: Deleting consensus metadata
I20260806 06:31:57.789510 22412 catalog_manager.cc:4841] TS 9fce1171e30449caa882cf50e03e6328 (127.7.43.67:34035): tablet 07d54b594c6a4ff39a8ea85801c64c9d (table test-table [id=d80f4e4076a740329f85ad6828409bb5]) successfully deleted
I20260806 06:31:57.789947 23177 ts_tablet_manager.cc:1905] T 07d54b594c6a4ff39a8ea85801c64c9d P 5ed71b7cbbae409bb5eb3929d40f6b10: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:31:57.791401 23177 ts_tablet_manager.cc:1918] T 07d54b594c6a4ff39a8ea85801c64c9d P 5ed71b7cbbae409bb5eb3929d40f6b10: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:31:57.791469 23177 log.cc:1196] T 07d54b594c6a4ff39a8ea85801c64c9d P 5ed71b7cbbae409bb5eb3929d40f6b10: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785997861828271-7341-0/minicluster-data/ts-1/wal/wals/07d54b594c6a4ff39a8ea85801c64c9d
I20260806 06:31:57.791716 23177 ts_tablet_manager.cc:1939] T 07d54b594c6a4ff39a8ea85801c64c9d P 5ed71b7cbbae409bb5eb3929d40f6b10: Deleting consensus metadata
I20260806 06:31:57.792102 22417 catalog_manager.cc:4841] TS 5ed71b7cbbae409bb5eb3929d40f6b10 (127.7.43.66:37525): tablet 07d54b594c6a4ff39a8ea85801c64c9d (table test-table [id=d80f4e4076a740329f85ad6828409bb5]) successfully deleted
W20260806 06:31:57.794519 22412 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.43.65:39657 (error 108)
I20260806 06:31:57.794951 23134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:31:57.794835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:39657 (local address 127.0.0.1:34340)
0806 06:31:57.794882 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 06:31:57.794919 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.65:39657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 06:31:57.794996 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 22732
W20260806 06:31:57.795063 22412 catalog_manager.cc:4584] TS c37f7a263f934e6192c0cf9c743467b6 (127.7.43.65:39657): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 07d54b594c6a4ff39a8ea85801c64c9d: Network error: Client connection negotiation failed: client connection to 127.7.43.65:39657: connect: Connection refused (error 111)
I20260806 06:31:57.806844 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 22925
I20260806 06:31:57.814122 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 22392
2026-08-06T06:31:57Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2129 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:31:57 dist-test-slave-6zbq krb5kdc[23183](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:31:57 dist-test-slave-6zbq krb5kdc[23183](info): set up 2 sockets
Aug 06 06:31:57 dist-test-slave-6zbq krb5kdc[23183](info): commencing operation
krb5kdc: starting...
W20260806 06:31:58.923960 7341 mini_kdc.cc:121] Time spent starting KDC: real 1.088s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Aug 06 06:31:58 dist-test-slave-6zbq krb5kdc[23183](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997918, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:31:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:31:59Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.7.43.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.7.43.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.7.43.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/krb5kdc/rangeradmin_127.7.43.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.7.43.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/krb5kdc/rangeradmin_127.7.43.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.7.43.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.7.43.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.7.43.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/krb5kdc/rangerlookup_127.7.43.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.7.43.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/krb5kdc/rangerlookup_127.7.43.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.43.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.7.43.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.43.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/krb5kdc/HTTP_127.7.43.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.43.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/krb5kdc/HTTP_127.7.43.84@KRBTEST.COM.keytab.
I20260806 06:31:59.066665 7341 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskfvM9Wt/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/postgres -l logfile start
2026-08-06 06:32:01.179 UTC [23230] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 06:32:01.179 UTC [23230] LOG: listening on IPv4 address "127.7.43.84", port 35013
2026-08-06 06:32:01.182 UTC [23230] LOG: listening on Unix socket "/tmp/.s.PGSQL.35013"
2026-08-06 06:32:01.208 UTC [23236] LOG: database system was shut down at 2026-08-06 06:32:00 UTC
2026-08-06 06:32:01.212 UTC [23230] LOG: database system is ready to accept connections
I20260806 06:32:02.238734 7341 mini_postgres.cc:96] Postgres bound to 35013
2026-08-06 06:32:02.246 UTC [23253] FATAL: database "slave" does not exist
127.7.43.84:35013 - accepting connections
I20260806 06:32:02.247673 7341 mini_ranger.cc:161] Starting Ranger...
I20260806 06:32:02.262800 7341 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 06:32:02.625504 7341 mini_ranger.cc:87] Created ranger Postgres database
I20260806 06:32:02.625571 7341 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ranger-admin
2026-08-06 06:32:02,946 [I] DB FLAVOR :POSTGRES
2026-08-06 06:32:02,947 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 06:32:02,947 [I] Checking connection..
2026-08-06 06:32:02,947 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvM9Wt/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvM9Wt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.43.84:35013/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 06:32:03,209 [I] Checking connection passed.
2026-08-06 06:32:03,209 [I] --------- Verifying version history table ---------
2026-08-06 06:32:03,210 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvM9Wt/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvM9Wt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.43.84:35013/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 06:32:03,454 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 06:32:03,454 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvM9Wt/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvM9Wt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.43.84:35013/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 06:32:03,683 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 06:32:03,683 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 06:32:03,684 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvM9Wt/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvM9Wt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.43.84:35013/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskfvM9Wt/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 06:32:03.928 UTC [23341] WARNING: there is no transaction in progress
2026-08-06 06:32:03,934 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 06:32:03,934 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvM9Wt/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvM9Wt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.43.84:35013/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 06:32:04,169 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 06:32:04,169 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 06:32:04,170 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvM9Wt/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvM9Wt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.43.84:35013/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 06:32:04,397 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvM9Wt/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvM9Wt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.43.84:35013/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 06:32:04,638 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvM9Wt/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvM9Wt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.43.84:35013/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-6zbq.c.gcp-upstream.internal','N') ;"
2026-08-06 06:32:04,869 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvM9Wt/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvM9Wt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.43.84:35013/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 06:32:05,119 [I] Table x_portal_user does not exist in database ranger
2026-08-06 06:32:05,120 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvM9Wt/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvM9Wt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.43.84:35013/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 06:32:05,360 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 06:32:05,360 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvM9Wt/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvM9Wt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.43.84:35013/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 06:32:05,603 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvM9Wt/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvM9Wt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.43.84:35013/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 06:32:05,849 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 06:32:05,849 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvM9Wt/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvM9Wt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.43.84:35013/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskfvM9Wt/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 06:32:06.724 UTC [23503] WARNING: there is no transaction in progress
2026-08-06 06:32:06.739 UTC [23503] WARNING: there is no transaction in progress
2026-08-06 06:32:06.754 UTC [23503] WARNING: there is no transaction in progress
2026-08-06 06:32:06.767 UTC [23503] WARNING: there is no transaction in progress
2026-08-06 06:32:06.781 UTC [23503] WARNING: there is no transaction in progress
2026-08-06 06:32:06.797 UTC [23503] WARNING: there is no transaction in progress
2026-08-06 06:32:06.847 UTC [23503] WARNING: there is no transaction in progress
2026-08-06 06:32:06.853 UTC [23503] WARNING: there is no transaction in progress
2026-08-06 06:32:06.866 UTC [23503] WARNING: there is no transaction in progress
2026-08-06 06:32:06.875 UTC [23503] WARNING: there is no transaction in progress
2026-08-06 06:32:06.885 UTC [23503] WARNING: there is no transaction in progress
2026-08-06 06:32:06.897 UTC [23503] WARNING: there is no transaction in progress
2026-08-06 06:32:06.906 UTC [23503] WARNING: there is no transaction in progress
2026-08-06 06:32:06.915 UTC [23503] WARNING: there is no transaction in progress
2026-08-06 06:32:06.923 UTC [23503] WARNING: there is no transaction in progress
2026-08-06 06:32:06.932 UTC [23503] WARNING: there is no transaction in progress
2026-08-06 06:32:07,794 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 06:32:07,795 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvM9Wt/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvM9Wt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.43.84:35013/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 06:32:08,029 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 06:32:08,030 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvM9Wt/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvM9Wt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.43.84:35013/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 06:32:08,254 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 06:32:08,255 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvM9Wt/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvM9Wt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.43.84:35013/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 06:32:08,503 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 06:32:08,503 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvM9Wt/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvM9Wt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.43.84:35013/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 06:32:08,745 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvM9Wt/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvM9Wt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.43.84:35013/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 06:32:08,968 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvM9Wt/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvM9Wt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.43.84:35013/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-6zbq.c.gcp-upstream.internal';"
2026-08-06 06:32:09,212 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 06:32:09,212 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvM9Wt/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvM9Wt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.43.84:35013/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 06:32:09,462 [I] DB_PATCHES have already been applied
I20260806 06:32:09.467823 7341 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ranger-admin:/tmp/dist-test-taskfvM9Wt/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskfvM9Wt/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskfvM9Wt/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvM9Wt/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 06:32:09.467880 7341 mini_ranger.cc:193] Using host: 127.7.43.84
I20260806 06:32:09.471324 7341 mini_ranger.cc:227] Ranger admin URL: http://127.7.43.84:46621
Aug 06, 2026 6:32:09 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 6:32:09 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 6:32:09 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 6:32:09 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 6:32:09 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 6:32:10 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 6:32:10 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 06:32:10.655879 7341 mini_ranger.cc:160] Time spent starting Ranger: real 8.408s user 0.006s sys 0.003s
Aug 06, 2026 6:32:29 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 6:32:29 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 06, 2026 6:32:29 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 6:32:29 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 6:32:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 6:32:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 6:32:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 6:32:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 6:32:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 6:32:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 6:32:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 6:32:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 6:32:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 6:32:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 6:32:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 6:32:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 6:32:29 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 6:32:30 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 06:32:31.235232 7341 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.7.43.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.7.43.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.7.43.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/krb5kdc/rangerkms_127.7.43.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.7.43.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/krb5kdc/rangerkms_127.7.43.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.43.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.7.43.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.43.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/krb5kdc/HTTP_127.7.43.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.43.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/krb5kdc/HTTP_127.7.43.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 06:32:31 dist-test-slave-6zbq krb5kdc[23183](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997951, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 06:32:31.311218 7341 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 06:32:31.322568 7341 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 06:32:32.012275 7341 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 06:32:32.012342 7341 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ranger-kms
I20260806 06:32:32.012358 7341 mini_ranger_kms.cc:227] Using postgres at 127.7.43.84:35013
2026-08-06 06:32:32,441 [I] DB FLAVOR :POSTGRES
2026-08-06 06:32:32,441 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 06:32:32,441 [I] Checking connection
2026-08-06 06:32:32,441 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvM9Wt/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.43.84:35013/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 06:32:32,753 [I] connection success
2026-08-06 06:32:32,753 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 06:32:32,754 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 06:32:32,754 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvM9Wt/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.43.84:35013/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 06:32:32,999 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 06:32:32,999 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 06:32:32,999 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 06:32:33,000 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvM9Wt/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.43.84:35013/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 06:32:33,251 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 06:32:33.686496 7341 mini_ranger_kms.cc:310] Created kms service
I20260806 06:32:33.880730 7341 mini_ranger_kms.cc:326] Created kudu user
I20260806 06:32:33.919883 7341 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 06:32:33 dist-test-slave-6zbq krb5kdc[23183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997951, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.7.43.84@KRBTEST.COM
I20260806 06:32:34.298084 7341 mini_ranger_kms.cc:384] Added ranger policy
I20260806 06:32:34.298233 7341 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ranger-kms:/tmp/dist-test-taskfvM9Wt/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvM9Wt/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskfvM9Wt/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 06:32:34.298256 7341 mini_ranger_kms.cc:240] Using host: 127.7.43.84
I20260806 06:32:34.302052 7341 mini_ranger_kms.cc:276] Ranger KMS PID: 23793
I20260806 06:32:34.302170 7341 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.7.43.84:53109
Aug 06, 2026 6:32:35 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 6:32:35 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 6:32:35 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 6:32:35 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 6:32:35 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 6:32:35 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260806 06:32:36.744112 7341 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.433s user 0.007s sys 0.284s
I20260806 06:32:36.744225 7341 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 06:32:36.744302 7341 mini_ranger_kms.cc:401] 127.7.43.84:53109/kms/v1/keys
Aug 06 06:32:36 dist-test-slave-6zbq krb5kdc[23183](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785997956, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.43.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 06:32:37 dist-test-slave-6zbq krb5kdc[23183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997956, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.43.84@KRBTEST.COM for HTTP/127.7.43.84@KRBTEST.COM
Aug 06 06:32:37 dist-test-slave-6zbq krb5kdc[23183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997956, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.43.84@KRBTEST.COM for HTTP/127.7.43.84@KRBTEST.COM
Aug 06 06:32:39 dist-test-slave-6zbq krb5kdc[23183](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997959, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.7.43.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
I20260806 06:32:39.912096 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:33613
--webserver_interface=127.7.43.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:34765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.43.126:33613
--ranger_config_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.43.84:53109/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:32:39.993537 23865 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:32:39.993710 23865 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:32:39.993731 23865 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:32:39.994844 23865 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:32:39.994882 23865 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:32:39.994895 23865 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:32:39.994906 23865 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:32:39.994918 23865 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:32:39.996461 23865 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:34765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/master-0/wal
--ranger_kms_url=127.7.43.84:53109/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.43.126:33613
--ranger_config_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:33613
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.43.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23865
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:32:39.996663 23865 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:32:39.996856 23865 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:32:40.001454 23878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:32:40.001498 23865 server_base.cc:1034] running on GCE node
W20260806 06:32:40.001629 23880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:32:40.001469 23875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:32:40.001902 23865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:32:40.002158 23865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:32:40.003368 23865 hybrid_clock.cc:648] HybridClock initialized: now 1785997960003332 us; error 40 us; skew 500 ppm
Aug 06 06:32:39 dist-test-slave-6zbq krb5kdc[23183](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997959, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:32:40.005898 23865 init.cc:377] Logged in from keytab as kudu/127.7.43.126@KRBTEST.COM (short username kudu)
I20260806 06:32:40.006282 23865 webserver.cc:460] Webserver started at http://127.7.43.126:38731/ using document root <none> and password file <none>
I20260806 06:32:40.006484 23865 fs_manager.cc:359] Metadata directory not provided
I20260806 06:32:40.006525 23865 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:32:40.006655 23865 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 06:32:40 dist-test-slave-6zbq krb5kdc[23183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997959, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.126@KRBTEST.COM for HTTP/127.7.43.84@KRBTEST.COM
I20260806 06:32:40.050393 23865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/master-0/data/instance:
uuid: "62664b2dd10242a6954edf80774433bf"
format_stamp: "Formatted at 2026-08-06 06:32:40 on dist-test-slave-6zbq"
server_key: "63c7744939a9a54053d657377e627d50"
server_key_iv: "be96dfb76d3ad72fc0b31b6c10d71e34"
server_key_version: "kuduclusterkey@0"
I20260806 06:32:40.050830 23865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/master-0/wal/instance:
uuid: "62664b2dd10242a6954edf80774433bf"
format_stamp: "Formatted at 2026-08-06 06:32:40 on dist-test-slave-6zbq"
server_key: "63c7744939a9a54053d657377e627d50"
server_key_iv: "be96dfb76d3ad72fc0b31b6c10d71e34"
server_key_version: "kuduclusterkey@0"
I20260806 06:32:40.052479 23865 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:32:40.066998 23889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:32:40.067191 23865 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.000s
I20260806 06:32:40.067296 23865 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "62664b2dd10242a6954edf80774433bf"
format_stamp: "Formatted at 2026-08-06 06:32:40 on dist-test-slave-6zbq"
server_key: "63c7744939a9a54053d657377e627d50"
server_key_iv: "be96dfb76d3ad72fc0b31b6c10d71e34"
server_key_version: "kuduclusterkey@0"
I20260806 06:32:40.067379 23865 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:32:40.084257 23865 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:32:40.086078 23865 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:32:40.086225 23865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:32:40.096777 23865 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.126:33613
I20260806 06:32:40.096957 23962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.126:33613 every 8 connection(s)
I20260806 06:32:40.097276 23865 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:32:40.098225 23963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:32:40.101244 23963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62664b2dd10242a6954edf80774433bf: Bootstrap starting.
I20260806 06:32:40.102205 23963 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 62664b2dd10242a6954edf80774433bf: Neither blocks nor log segments found. Creating new log.
I20260806 06:32:40.102864 23963 log.cc:824] T 00000000000000000000000000000000 P 62664b2dd10242a6954edf80774433bf: Log is configured to *not* fsync() on all Append() calls
I20260806 06:32:40.104377 23963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62664b2dd10242a6954edf80774433bf: No bootstrap required, opened a new log
I20260806 06:32:40.106964 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 23865
I20260806 06:32:40.106930 23963 raft_consensus.cc:348] T 00000000000000000000000000000000 P 62664b2dd10242a6954edf80774433bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62664b2dd10242a6954edf80774433bf" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 33613 } }
I20260806 06:32:40.107067 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/master-0/wal/instance
I20260806 06:32:40.107071 23963 raft_consensus.cc:374] T 00000000000000000000000000000000 P 62664b2dd10242a6954edf80774433bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:32:40.107105 23963 raft_consensus.cc:729] T 00000000000000000000000000000000 P 62664b2dd10242a6954edf80774433bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62664b2dd10242a6954edf80774433bf, State: Initialized, Role: FOLLOWER
I20260806 06:32:40.107257 7341 external_mini_cluster.cc:1428] Setting key 49ed5e6313838f6a79fc7d1d5448577a
I20260806 06:32:40.107256 23963 consensus_queue.cc:260] T 00000000000000000000000000000000 P 62664b2dd10242a6954edf80774433bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62664b2dd10242a6954edf80774433bf" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 33613 } }
I20260806 06:32:40.107327 23963 raft_consensus.cc:388] T 00000000000000000000000000000000 P 62664b2dd10242a6954edf80774433bf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:32:40.107379 23963 raft_consensus.cc:482] T 00000000000000000000000000000000 P 62664b2dd10242a6954edf80774433bf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:32:40.107445 23963 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 62664b2dd10242a6954edf80774433bf [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:32:40.108089 23963 raft_consensus.cc:504] T 00000000000000000000000000000000 P 62664b2dd10242a6954edf80774433bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62664b2dd10242a6954edf80774433bf" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 33613 } }
I20260806 06:32:40.108201 23963 leader_election.cc:302] T 00000000000000000000000000000000 P 62664b2dd10242a6954edf80774433bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 62664b2dd10242a6954edf80774433bf; no voters:
I20260806 06:32:40.108398 23963 leader_election.cc:288] T 00000000000000000000000000000000 P 62664b2dd10242a6954edf80774433bf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:32:40.108474 23970 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 62664b2dd10242a6954edf80774433bf [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:32:40.108723 23963 sys_catalog.cc:564] T 00000000000000000000000000000000 P 62664b2dd10242a6954edf80774433bf [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:32:40.108790 23970 raft_consensus.cc:686] T 00000000000000000000000000000000 P 62664b2dd10242a6954edf80774433bf [term 1 LEADER]: Becoming Leader. State: Replica: 62664b2dd10242a6954edf80774433bf, State: Running, Role: LEADER
I20260806 06:32:40.108958 23970 consensus_queue.cc:237] T 00000000000000000000000000000000 P 62664b2dd10242a6954edf80774433bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62664b2dd10242a6954edf80774433bf" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 33613 } }
I20260806 06:32:40.109301 23970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62664b2dd10242a6954edf80774433bf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "62664b2dd10242a6954edf80774433bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62664b2dd10242a6954edf80774433bf" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 33613 } } }
I20260806 06:32:40.109375 23970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62664b2dd10242a6954edf80774433bf [sys.catalog]: This master's current role is: LEADER
I20260806 06:32:40.109408 23972 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62664b2dd10242a6954edf80774433bf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 62664b2dd10242a6954edf80774433bf. Latest consensus state: current_term: 1 leader_uuid: "62664b2dd10242a6954edf80774433bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62664b2dd10242a6954edf80774433bf" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 33613 } } }
I20260806 06:32:40.109511 23972 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62664b2dd10242a6954edf80774433bf [sys.catalog]: This master's current role is: LEADER
I20260806 06:32:40.109781 23975 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:32:40.110359 23975 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:32:40.112116 23975 catalog_manager.cc:1347] Generated new cluster ID: a76923cd658c424bb315537803505af5
I20260806 06:32:40.112165 23975 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:32:40.112672 23963 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
Aug 06 06:32:40 dist-test-slave-6zbq krb5kdc[23183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997959, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:32:40.120807 23975 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:32:40.121047 23969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:32:40.107840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50540 (local address 127.7.43.126:33613)
0806 06:32:40.108145 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0806 06:32:40.108154 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:32:40.108210 (+ 56us) server_negotiation.cc:407] Connection header received
0806 06:32:40.108334 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:32:40.108341 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:32:40.108398 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:32:40.109147 (+ 749us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:32:40.109431 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:32:40.111372 (+ 1941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:32:40.113136 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:32:40.113685 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:32:40.113838 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:32:40.116607 (+ 2769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:32:40.116624 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:32:40.116624 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:32:40.116636 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:32:40.119073 (+ 2437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:32:40.119475 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:32:40.119478 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:32:40.119478 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:32:40.119589 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:32:40.119823 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:32:40.119824 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:32:40.119824 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:32:40.120122 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:32:40.120646 (+ 524us) server_negotiation.cc:1035] Waiting for connection context
0806 06:32:40.120719 (+ 73us) server_negotiation.cc:299] Negotiation successful
0806 06:32:40.120835 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":235,"thread_start_us":154,"threads_started":1}
I20260806 06:32:40.121536 23975 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:32:40.131178 23975 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 62664b2dd10242a6954edf80774433bf: Generated new TSK 0
I20260806 06:32:40.131451 23975 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.43.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab.
Entry for principal kudu/127.7.43.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab.
I20260806 06:32:41.000874 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.43.84:53109/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.65:0
--local_ip_for_outbound_sockets=127.7.43.65
--webserver_interface=127.7.43.65
--webserver_port=0
--tserver_master_addrs=127.7.43.126:33613
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab
--principal=kudu/127.7.43.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:34765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:32:41.098266 24027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:32:41.098552 24027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:32:41.098584 24027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:32:41.099895 24027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:32:41.099941 24027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:32:41.099964 24027 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.65
I20260806 06:32:41.101622 24027 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:34765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.7.43.84:53109/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab
--principal=kudu/127.7.43.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.43.65
--webserver_port=0
--tserver_master_addrs=127.7.43.126:33613
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24027
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.65
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:32:41.101824 24027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:32:41.102053 24027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:32:41.107187 24040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:32:41.107280 24042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:32:41.107367 24039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:32:41.108057 24027 server_base.cc:1034] running on GCE node
I20260806 06:32:41.108242 24027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:32:41.108448 24027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:32:41.109586 24027 hybrid_clock.cc:648] HybridClock initialized: now 1785997961109574 us; error 34 us; skew 500 ppm
Aug 06 06:32:41 dist-test-slave-6zbq krb5kdc[23183](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997961, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:32:41.112226 24027 init.cc:377] Logged in from keytab as kudu/127.7.43.65@KRBTEST.COM (short username kudu)
I20260806 06:32:41.112641 24027 webserver.cc:460] Webserver started at http://127.7.43.65:43043/ using document root <none> and password file <none>
I20260806 06:32:41.112882 24027 fs_manager.cc:359] Metadata directory not provided
I20260806 06:32:41.112924 24027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:32:41.113025 24027 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 06:32:41 dist-test-slave-6zbq krb5kdc[23183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997961, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.65@KRBTEST.COM for HTTP/127.7.43.84@KRBTEST.COM
I20260806 06:32:41.126675 24027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ts-0/data/instance:
uuid: "bb30337f6b5649df8c3b08d4c616cdca"
format_stamp: "Formatted at 2026-08-06 06:32:41 on dist-test-slave-6zbq"
server_key: "809755b9efbd8d748d58b32b9029e971"
server_key_iv: "6f482d04505f3e2f019df65288dc1cb8"
server_key_version: "kuduclusterkey@0"
I20260806 06:32:41.127075 24027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ts-0/wal/instance:
uuid: "bb30337f6b5649df8c3b08d4c616cdca"
format_stamp: "Formatted at 2026-08-06 06:32:41 on dist-test-slave-6zbq"
server_key: "809755b9efbd8d748d58b32b9029e971"
server_key_iv: "6f482d04505f3e2f019df65288dc1cb8"
server_key_version: "kuduclusterkey@0"
I20260806 06:32:41.128695 24027 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:32:41.142724 24052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:32:41.143356 24027 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:32:41.143461 24027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ts-0/wal
uuid: "bb30337f6b5649df8c3b08d4c616cdca"
format_stamp: "Formatted at 2026-08-06 06:32:41 on dist-test-slave-6zbq"
server_key: "809755b9efbd8d748d58b32b9029e971"
server_key_iv: "6f482d04505f3e2f019df65288dc1cb8"
server_key_version: "kuduclusterkey@0"
I20260806 06:32:41.143548 24027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:32:41.176764 24027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:32:41.179060 24027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:32:41.179243 24027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:32:41.179463 24027 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:32:41.179795 24027 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:32:41.179831 24027 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:32:41.179859 24027 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:32:41.179898 24027 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:32:41.187685 24027 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.65:38943
I20260806 06:32:41.187824 24209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.65:38943 every 8 connection(s)
I20260806 06:32:41.188074 24027 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
I20260806 06:32:41.192196 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 24027
I20260806 06:32:41.192384 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ts-0/wal/instance
I20260806 06:32:41.192766 7341 external_mini_cluster.cc:1428] Setting key aabd7f93c597a75ea7729901ba03c35b
Aug 06 06:32:41 dist-test-slave-6zbq krb5kdc[23183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997961, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.65@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.43.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.66@KRBTEST.COM" created.
I20260806 06:32:41.209282 24213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:32:41.188902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:37377 (local address 127.7.43.126:33613)
0806 06:32:41.193303 (+ 4401us) server_negotiation.cc:206] Beginning negotiation
0806 06:32:41.193305 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:32:41.193315 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:32:41.193343 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:32:41.193343 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:32:41.193357 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:32:41.198517 (+ 5160us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:32:41.200355 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:32:41.201195 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:32:41.203508 (+ 2313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:32:41.203885 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:32:41.205199 (+ 1314us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:32:41.206023 (+ 824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:32:41.206025 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:32:41.206025 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:32:41.206032 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:32:41.207836 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:32:41.208208 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:32:41.208210 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:32:41.208210 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:32:41.208291 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:32:41.208609 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:32:41.208610 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:32:41.208610 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:32:41.208711 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:32:41.208772 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 06:32:41.209083 (+ 311us) server_negotiation.cc:299] Negotiation successful
0806 06:32:41.209171 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4376,"thread_start_us":61,"threads_started":1}
I20260806 06:32:41.209484 24212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:32:41.188858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33613 (local address 127.7.43.65:37377)
0806 06:32:41.189100 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0806 06:32:41.189138 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 06:32:41.189768 (+ 630us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:32:41.199720 (+ 9952us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:32:41.199737 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:32:41.199984 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:32:41.200260 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:32:41.200266 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:32:41.201288 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:32:41.201289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:32:41.203411 (+ 2122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:32:41.203414 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:32:41.203992 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:32:41.203993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:32:41.204156 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:32:41.204721 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:32:41.204742 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:32:41.205935 (+ 1193us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:32:41.207909 (+ 1974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:32:41.207916 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:32:41.207918 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:32:41.208143 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:32:41.208388 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:32:41.208389 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:32:41.208389 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:32:41.208543 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:32:41.208782 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:32:41.208789 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:32:41.208956 (+ 167us) client_negotiation.cc:769] Sending connection context
0806 06:32:41.209052 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 06:32:41.209197 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":85,"threads_started":1}
I20260806 06:32:41.210048 24210 heartbeater.cc:347] Connected to a master server at 127.7.43.126:33613
I20260806 06:32:41.210143 24210 heartbeater.cc:464] Registering TS with master...
I20260806 06:32:41.210319 24210 heartbeater.cc:511] Master 127.7.43.126:33613 requested a full tablet report, sending...
I20260806 06:32:41.210909 23916 ts_manager.cc:194] Registered new tserver with Master: bb30337f6b5649df8c3b08d4c616cdca (127.7.43.65:38943)
I20260806 06:32:41.211715 23916 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.65@KRBTEST.COM'} at 127.7.43.65:37377
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab.
Entry for principal kudu/127.7.43.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab.
I20260806 06:32:41.223414 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.43.84:53109/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.66:0
--local_ip_for_outbound_sockets=127.7.43.66
--webserver_interface=127.7.43.66
--webserver_port=0
--tserver_master_addrs=127.7.43.126:33613
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab
--principal=kudu/127.7.43.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:34765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 06:32:41 dist-test-slave-6zbq krb5kdc[23183](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785997961, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 06:32:41.350978 24217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:32:41.351246 24217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:32:41.351286 24217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:32:41.352893 24217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:32:41.352954 24217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:32:41.352993 24217 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.66
I20260806 06:32:41.355305 24217 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:34765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.7.43.84:53109/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab
--principal=kudu/127.7.43.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.43.66
--webserver_port=0
--tserver_master_addrs=127.7.43.126:33613
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24217
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.66
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:32:41.355538 24217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:32:41.355798 24217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:32:41.361749 24229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:32:41.361946 24228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:32:41.361768 24231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:32:41.362251 24217 server_base.cc:1034] running on GCE node
I20260806 06:32:41.362468 24217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:32:41.362691 24217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:32:41.363849 24217 hybrid_clock.cc:648] HybridClock initialized: now 1785997961363837 us; error 26 us; skew 500 ppm
Aug 06 06:32:41 dist-test-slave-6zbq krb5kdc[23183](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997961, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:32:41.366240 24217 init.cc:377] Logged in from keytab as kudu/127.7.43.66@KRBTEST.COM (short username kudu)
I20260806 06:32:41.366679 24217 webserver.cc:460] Webserver started at http://127.7.43.66:37477/ using document root <none> and password file <none>
I20260806 06:32:41.366905 24217 fs_manager.cc:359] Metadata directory not provided
I20260806 06:32:41.366951 24217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:32:41.367082 24217 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 06:32:41 dist-test-slave-6zbq krb5kdc[23183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997961, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.66@KRBTEST.COM for HTTP/127.7.43.84@KRBTEST.COM
I20260806 06:32:41.388492 24217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ts-1/data/instance:
uuid: "f72f6d4758914628b0e6858beaa29ebf"
format_stamp: "Formatted at 2026-08-06 06:32:41 on dist-test-slave-6zbq"
server_key: "996ecaf4a9329e5396918b0db120a947"
server_key_iv: "0dbf3262152c34b70316635bb65ebc0c"
server_key_version: "kuduclusterkey@0"
I20260806 06:32:41.389083 24217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ts-1/wal/instance:
uuid: "f72f6d4758914628b0e6858beaa29ebf"
format_stamp: "Formatted at 2026-08-06 06:32:41 on dist-test-slave-6zbq"
server_key: "996ecaf4a9329e5396918b0db120a947"
server_key_iv: "0dbf3262152c34b70316635bb65ebc0c"
server_key_version: "kuduclusterkey@0"
I20260806 06:32:41.391062 24217 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260806 06:32:41.405200 24247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:32:41.409235 24217 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.000s
I20260806 06:32:41.409353 24217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ts-1/wal
uuid: "f72f6d4758914628b0e6858beaa29ebf"
format_stamp: "Formatted at 2026-08-06 06:32:41 on dist-test-slave-6zbq"
server_key: "996ecaf4a9329e5396918b0db120a947"
server_key_iv: "0dbf3262152c34b70316635bb65ebc0c"
server_key_version: "kuduclusterkey@0"
I20260806 06:32:41.409456 24217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:32:41.426419 24217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:32:41.428647 24217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:32:41.428798 24217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:32:41.429040 24217 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:32:41.429471 24217 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:32:41.429518 24217 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:32:41.429545 24217 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:32:41.429570 24217 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:32:41.438434 24217 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.66:41377
I20260806 06:32:41.439000 24217 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
I20260806 06:32:41.442931 24431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.66:41377 every 8 connection(s)
I20260806 06:32:41.446540 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 24217
I20260806 06:32:41.446653 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ts-1/wal/instance
I20260806 06:32:41.446839 7341 external_mini_cluster.cc:1428] Setting key b344e0de8318b479bcbba1279b0a836d
Aug 06 06:32:41 dist-test-slave-6zbq krb5kdc[23183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997961, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.66@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:32:41.457146 24213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:32:41.446004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:34181 (local address 127.7.43.126:33613)
0806 06:32:41.446059 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 06:32:41.446062 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:32:41.446390 (+ 328us) server_negotiation.cc:407] Connection header received
0806 06:32:41.447889 (+ 1499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:32:41.447890 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:32:41.447902 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:32:41.447930 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:32:41.448820 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:32:41.449554 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:32:41.451041 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:32:41.451332 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:32:41.451375 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:32:41.453992 (+ 2617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:32:41.453993 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:32:41.453994 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:32:41.454001 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:32:41.455627 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:32:41.456103 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:32:41.456104 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:32:41.456105 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:32:41.456190 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:32:41.456446 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:32:41.456447 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:32:41.456448 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:32:41.456569 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:32:41.456608 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0806 06:32:41.456962 (+ 354us) server_negotiation.cc:299] Negotiation successful
0806 06:32:41.457047 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 06:32:41.457847 24434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:32:41.445706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33613 (local address 127.7.43.66:34181)
0806 06:32:41.446221 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0806 06:32:41.446267 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 06:32:41.447773 (+ 1506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:32:41.448160 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:32:41.448164 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:32:41.448397 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:32:41.448671 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:32:41.448677 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:32:41.449688 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:32:41.449689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:32:41.450942 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:32:41.450945 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:32:41.451468 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:32:41.451469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:32:41.451697 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:32:41.452324 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:32:41.452334 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:32:41.453882 (+ 1548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:32:41.455724 (+ 1842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:32:41.455730 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:32:41.455733 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:32:41.456009 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:32:41.456256 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:32:41.456257 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:32:41.456257 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:32:41.456380 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:32:41.456652 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:32:41.456657 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:32:41.456821 (+ 164us) client_negotiation.cc:769] Sending connection context
0806 06:32:41.457472 (+ 651us) client_negotiation.cc:240] Negotiation successful
0806 06:32:41.457630 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":93,"threads_started":1}
I20260806 06:32:41.458586 24432 heartbeater.cc:347] Connected to a master server at 127.7.43.126:33613
I20260806 06:32:41.458678 24432 heartbeater.cc:464] Registering TS with master...
I20260806 06:32:41.458854 24432 heartbeater.cc:511] Master 127.7.43.126:33613 requested a full tablet report, sending...
I20260806 06:32:41.459265 23916 ts_manager.cc:194] Registered new tserver with Master: f72f6d4758914628b0e6858beaa29ebf (127.7.43.66:41377)
I20260806 06:32:41.459702 23916 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.66@KRBTEST.COM'} at 127.7.43.66:34181
WARNING: no policy specified for kudu/127.7.43.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab.
Entry for principal kudu/127.7.43.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab.
I20260806 06:32:41.485868 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.43.84:53109/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.67:0
--local_ip_for_outbound_sockets=127.7.43.67
--webserver_interface=127.7.43.67
--webserver_port=0
--tserver_master_addrs=127.7.43.126:33613
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab
--principal=kudu/127.7.43.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:34765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:32:41.642830 24438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:32:41.643045 24438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:32:41.643075 24438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:32:41.644652 24438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:32:41.644711 24438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:32:41.644748 24438 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.67
I20260806 06:32:41.646976 24438 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:34765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.7.43.84:53109/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab
--principal=kudu/127.7.43.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.43.67
--webserver_port=0
--tserver_master_addrs=127.7.43.126:33613
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24438
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.67
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:32:41.647223 24438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:32:41.647473 24438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:32:41.655045 24448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:32:41.655063 24451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:32:41.655462 24447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:32:41.659546 24438 server_base.cc:1034] running on GCE node
I20260806 06:32:41.659879 24438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:32:41.660143 24438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:32:41.663388 24438 hybrid_clock.cc:648] HybridClock initialized: now 1785997961663368 us; error 35 us; skew 500 ppm
Aug 06 06:32:41 dist-test-slave-6zbq krb5kdc[23183](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997961, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:32:41.666086 24438 init.cc:377] Logged in from keytab as kudu/127.7.43.67@KRBTEST.COM (short username kudu)
I20260806 06:32:41.666728 24438 webserver.cc:460] Webserver started at http://127.7.43.67:39851/ using document root <none> and password file <none>
I20260806 06:32:41.666967 24438 fs_manager.cc:359] Metadata directory not provided
I20260806 06:32:41.667022 24438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:32:41.667160 24438 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 06:32:41 dist-test-slave-6zbq krb5kdc[23183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997961, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.67@KRBTEST.COM for HTTP/127.7.43.84@KRBTEST.COM
I20260806 06:32:41.681201 24438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ts-2/data/instance:
uuid: "0e6b240031da49b690c00599ba37b872"
format_stamp: "Formatted at 2026-08-06 06:32:41 on dist-test-slave-6zbq"
server_key: "f5e0502cab29137cf3a1f8b76c5f6774"
server_key_iv: "b6be848058f034cb606da7c28eb283e7"
server_key_version: "kuduclusterkey@0"
I20260806 06:32:41.681747 24438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ts-2/wal/instance:
uuid: "0e6b240031da49b690c00599ba37b872"
format_stamp: "Formatted at 2026-08-06 06:32:41 on dist-test-slave-6zbq"
server_key: "f5e0502cab29137cf3a1f8b76c5f6774"
server_key_iv: "b6be848058f034cb606da7c28eb283e7"
server_key_version: "kuduclusterkey@0"
I20260806 06:32:41.683789 24438 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260806 06:32:41.704180 24463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:32:41.704378 24438 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.000s sys 0.001s
I20260806 06:32:41.704483 24438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ts-2/wal
uuid: "0e6b240031da49b690c00599ba37b872"
format_stamp: "Formatted at 2026-08-06 06:32:41 on dist-test-slave-6zbq"
server_key: "f5e0502cab29137cf3a1f8b76c5f6774"
server_key_iv: "b6be848058f034cb606da7c28eb283e7"
server_key_version: "kuduclusterkey@0"
I20260806 06:32:41.704599 24438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:32:41.740685 24438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:32:41.743458 24438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:32:41.743631 24438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:32:41.743999 24438 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:32:41.744462 24438 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:32:41.744520 24438 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:32:41.744558 24438 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:32:41.744586 24438 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:32:41.751595 24438 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.67:38285
I20260806 06:32:41.752170 24631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.67:38285 every 8 connection(s)
I20260806 06:32:41.752609 24438 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
I20260806 06:32:41.760164 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 24438
I20260806 06:32:41.760257 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ts-2/wal/instance
I20260806 06:32:41.760444 7341 external_mini_cluster.cc:1428] Setting key dfca7a0681033956d98bd29d46754d5e
Aug 06 06:32:41 dist-test-slave-6zbq krb5kdc[23183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997961, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.67@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:32:41.764240 24213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:32:41.753783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:55207 (local address 127.7.43.126:33613)
0806 06:32:41.753833 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 06:32:41.753836 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:32:41.753921 (+ 85us) server_negotiation.cc:407] Connection header received
0806 06:32:41.754679 (+ 758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:32:41.754680 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:32:41.754693 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:32:41.754768 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:32:41.755607 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:32:41.756378 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:32:41.757850 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:32:41.758166 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:32:41.758208 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:32:41.760978 (+ 2770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:32:41.760980 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:32:41.760980 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:32:41.760988 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:32:41.762632 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:32:41.763127 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:32:41.763128 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:32:41.763129 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:32:41.763190 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:32:41.763480 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:32:41.763481 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:32:41.763482 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:32:41.763618 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:32:41.763664 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 06:32:41.764053 (+ 389us) server_negotiation.cc:299] Negotiation successful
0806 06:32:41.764147 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 06:32:41.764914 24637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:32:41.753531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33613 (local address 127.7.43.67:55207)
0806 06:32:41.753800 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0806 06:32:41.753839 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 06:32:41.754555 (+ 716us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:32:41.754853 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:32:41.754858 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:32:41.755159 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:32:41.755498 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:32:41.755506 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:32:41.756493 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:32:41.756495 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:32:41.757745 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:32:41.757749 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:32:41.758297 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:32:41.758298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:32:41.758465 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:32:41.759136 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:32:41.759146 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:32:41.760882 (+ 1736us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:32:41.762728 (+ 1846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:32:41.762734 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:32:41.762737 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:32:41.763019 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:32:41.763266 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:32:41.763267 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:32:41.763267 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:32:41.763399 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:32:41.763713 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:32:41.763719 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:32:41.763889 (+ 170us) client_negotiation.cc:769] Sending connection context
0806 06:32:41.764479 (+ 590us) client_negotiation.cc:240] Negotiation successful
0806 06:32:41.764636 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":89,"threads_started":1}
I20260806 06:32:41.765412 24632 heartbeater.cc:347] Connected to a master server at 127.7.43.126:33613
I20260806 06:32:41.765511 24632 heartbeater.cc:464] Registering TS with master...
I20260806 06:32:41.765699 24632 heartbeater.cc:511] Master 127.7.43.126:33613 requested a full tablet report, sending...
I20260806 06:32:41.766057 23916 ts_manager.cc:194] Registered new tserver with Master: 0e6b240031da49b690c00599ba37b872 (127.7.43.67:38285)
I20260806 06:32:41.766574 23916 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.67@KRBTEST.COM'} at 127.7.43.67:55207
I20260806 06:32:41.772079 7341 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:32:41.793649 24213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:32:41.774817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50548 (local address 127.7.43.126:33613)
0806 06:32:41.775227 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0806 06:32:41.775229 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:32:41.782571 (+ 7342us) server_negotiation.cc:407] Connection header received
0806 06:32:41.783266 (+ 695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:32:41.783267 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:32:41.783279 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:32:41.783309 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:32:41.784183 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:32:41.784927 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:32:41.786327 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:32:41.786657 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:32:41.786705 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:32:41.788372 (+ 1667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:32:41.788374 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:32:41.788374 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:32:41.788382 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:32:41.790010 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:32:41.792719 (+ 2709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:32:41.792720 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:32:41.792720 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:32:41.792783 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:32:41.793013 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:32:41.793014 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:32:41.793014 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:32:41.793149 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:32:41.793227 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 06:32:41.793475 (+ 248us) server_negotiation.cc:299] Negotiation successful
0806 06:32:41.793553 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383}
I20260806 06:32:41.795794 23916 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50548:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:32:41.798300 23916 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:32:41.818511 24650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:32:41.811965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.67:38285 (local address 127.0.0.1:55740)
0806 06:32:41.813856 (+ 1891us) negotiation.cc:107] Waiting for socket to connect
0806 06:32:41.813865 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:32:41.813909 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:32:41.814196 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:32:41.814200 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:32:41.814202 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:32:41.814275 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:32:41.814277 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:32:41.815650 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:32:41.815651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:32:41.816927 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:32:41.816930 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:32:41.818176 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:32:41.818177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:32:41.818259 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:32:41.818275 (+ 16us) client_negotiation.cc:769] Sending connection context
0806 06:32:41.818367 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 06:32:41.818427 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1875,"thread_start_us":125,"threads_started":1}
I20260806 06:32:41.818998 24657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:32:41.812315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55740 (local address 127.7.43.67:38285)
0806 06:32:41.813258 (+ 943us) server_negotiation.cc:206] Beginning negotiation
0806 06:32:41.813264 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 06:32:41.813962 (+ 698us) server_negotiation.cc:407] Connection header received
0806 06:32:41.814010 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:32:41.814012 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:32:41.814030 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:32:41.814097 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:32:41.814376 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:32:41.815554 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:32:41.817146 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:32:41.817991 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:32:41.818063 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:32:41.818139 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 06:32:41.818873 (+ 734us) server_negotiation.cc:299] Negotiation successful
0806 06:32:41.818916 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":889,"thread_start_us":120,"threads_started":1}
I20260806 06:32:41.819831 24506 tablet_service.cc:1467] Processing CreateTablet for tablet 500f83b5e96944ba86d482fc5989e64c (DEFAULT_TABLE table=test-table [id=13264de7acff40f297f306089048bf0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:32:41.820202 24506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 500f83b5e96944ba86d482fc5989e64c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:32:41.824656 24658 tablet_bootstrap.cc:492] T 500f83b5e96944ba86d482fc5989e64c P 0e6b240031da49b690c00599ba37b872: Bootstrap starting.
I20260806 06:32:41.825688 24658 tablet_bootstrap.cc:654] T 500f83b5e96944ba86d482fc5989e64c P 0e6b240031da49b690c00599ba37b872: Neither blocks nor log segments found. Creating new log.
I20260806 06:32:41.826370 24658 log.cc:824] T 500f83b5e96944ba86d482fc5989e64c P 0e6b240031da49b690c00599ba37b872: Log is configured to *not* fsync() on all Append() calls
I20260806 06:32:41.827239 24658 tablet_bootstrap.cc:492] T 500f83b5e96944ba86d482fc5989e64c P 0e6b240031da49b690c00599ba37b872: No bootstrap required, opened a new log
I20260806 06:32:41.827301 24658 ts_tablet_manager.cc:1397] T 500f83b5e96944ba86d482fc5989e64c P 0e6b240031da49b690c00599ba37b872: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 06:32:41.827755 24652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:32:41.811703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.66:41377 (local address 127.0.0.1:52640)
0806 06:32:41.813675 (+ 1972us) negotiation.cc:107] Waiting for socket to connect
0806 06:32:41.813684 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:32:41.813724 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:32:41.822696 (+ 8972us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:32:41.822697 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:32:41.822700 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:32:41.822743 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:32:41.822744 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:32:41.824281 (+ 1537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:32:41.824282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:32:41.825358 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:32:41.825361 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:32:41.827480 (+ 2119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:32:41.827481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:32:41.827545 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:32:41.827550 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:32:41.827584 (+ 34us) client_negotiation.cc:240] Negotiation successful
0806 06:32:41.827673 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1940,"mutex_wait_us":1,"thread_start_us":111,"threads_started":1}
I20260806 06:32:41.828161 24654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:32:41.811838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52640 (local address 127.7.43.66:41377)
0806 06:32:41.822445 (+ 10607us) server_negotiation.cc:206] Beginning negotiation
0806 06:32:41.822450 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:32:41.822462 (+ 12us) server_negotiation.cc:407] Connection header received
0806 06:32:41.822502 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:32:41.822505 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:32:41.822521 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:32:41.822606 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:32:41.822824 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:32:41.823879 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:32:41.825486 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:32:41.826230 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:32:41.827923 (+ 1693us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:32:41.828011 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 06:32:41.828040 (+ 29us) server_negotiation.cc:299] Negotiation successful
0806 06:32:41.828078 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10565,"thread_start_us":99,"threads_started":1}
I20260806 06:32:41.828770 24321 tablet_service.cc:1467] Processing CreateTablet for tablet 500f83b5e96944ba86d482fc5989e64c (DEFAULT_TABLE table=test-table [id=13264de7acff40f297f306089048bf0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:32:41.829118 24321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 500f83b5e96944ba86d482fc5989e64c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:32:41.830235 24658 raft_consensus.cc:348] T 500f83b5e96944ba86d482fc5989e64c P 0e6b240031da49b690c00599ba37b872 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb30337f6b5649df8c3b08d4c616cdca" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 38943 } } peers { permanent_uuid: "f72f6d4758914628b0e6858beaa29ebf" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 41377 } } peers { permanent_uuid: "0e6b240031da49b690c00599ba37b872" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 38285 } }
I20260806 06:32:41.830421 24658 raft_consensus.cc:374] T 500f83b5e96944ba86d482fc5989e64c P 0e6b240031da49b690c00599ba37b872 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:32:41.830454 24658 raft_consensus.cc:729] T 500f83b5e96944ba86d482fc5989e64c P 0e6b240031da49b690c00599ba37b872 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e6b240031da49b690c00599ba37b872, State: Initialized, Role: FOLLOWER
I20260806 06:32:41.831079 24658 consensus_queue.cc:260] T 500f83b5e96944ba86d482fc5989e64c P 0e6b240031da49b690c00599ba37b872 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb30337f6b5649df8c3b08d4c616cdca" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 38943 } } peers { permanent_uuid: "f72f6d4758914628b0e6858beaa29ebf" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 41377 } } peers { permanent_uuid: "0e6b240031da49b690c00599ba37b872" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 38285 } }
I20260806 06:32:41.831447 24658 ts_tablet_manager.cc:1428] T 500f83b5e96944ba86d482fc5989e64c P 0e6b240031da49b690c00599ba37b872: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260806 06:32:41.831610 24632 heartbeater.cc:503] Master 127.7.43.126:33613 was elected leader, sending a full tablet report...
I20260806 06:32:41.835909 24656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:32:41.811340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:38943 (local address 127.0.0.1:53824)
0806 06:32:41.813345 (+ 2005us) negotiation.cc:107] Waiting for socket to connect
0806 06:32:41.813429 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0806 06:32:41.813504 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:32:41.824405 (+ 10901us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:32:41.824406 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:32:41.824408 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:32:41.824463 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:32:41.824465 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:32:41.832994 (+ 8529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:32:41.832995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:32:41.834073 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:32:41.834077 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:32:41.835701 (+ 1624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:32:41.835702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:32:41.835768 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:32:41.835772 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:32:41.835801 (+ 29us) client_negotiation.cc:240] Negotiation successful
0806 06:32:41.835837 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1955,"mutex_wait_us":1,"thread_start_us":126,"threads_started":1}
I20260806 06:32:41.836241 24651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:32:41.811430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53824 (local address 127.7.43.65:38943)
0806 06:32:41.824024 (+ 12594us) server_negotiation.cc:206] Beginning negotiation
0806 06:32:41.824028 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:32:41.824038 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:32:41.824070 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:32:41.824072 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:32:41.824089 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:32:41.824167 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:32:41.826323 (+ 2156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:32:41.827365 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:32:41.834669 (+ 7304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:32:41.835600 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:32:41.836028 (+ 428us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:32:41.836096 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 06:32:41.836122 (+ 26us) server_negotiation.cc:299] Negotiation successful
0806 06:32:41.836162 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12554,"thread_start_us":43,"threads_started":1}
I20260806 06:32:41.836920 24123 tablet_service.cc:1467] Processing CreateTablet for tablet 500f83b5e96944ba86d482fc5989e64c (DEFAULT_TABLE table=test-table [id=13264de7acff40f297f306089048bf0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:32:41.837272 24123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 500f83b5e96944ba86d482fc5989e64c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:32:41.842298 24662 tablet_bootstrap.cc:492] T 500f83b5e96944ba86d482fc5989e64c P bb30337f6b5649df8c3b08d4c616cdca: Bootstrap starting.
I20260806 06:32:41.843777 24662 tablet_bootstrap.cc:654] T 500f83b5e96944ba86d482fc5989e64c P bb30337f6b5649df8c3b08d4c616cdca: Neither blocks nor log segments found. Creating new log.
I20260806 06:32:41.844540 24662 log.cc:824] T 500f83b5e96944ba86d482fc5989e64c P bb30337f6b5649df8c3b08d4c616cdca: Log is configured to *not* fsync() on all Append() calls
I20260806 06:32:41.844969 24661 tablet_bootstrap.cc:492] T 500f83b5e96944ba86d482fc5989e64c P f72f6d4758914628b0e6858beaa29ebf: Bootstrap starting.
I20260806 06:32:41.846655 24661 tablet_bootstrap.cc:654] T 500f83b5e96944ba86d482fc5989e64c P f72f6d4758914628b0e6858beaa29ebf: Neither blocks nor log segments found. Creating new log.
I20260806 06:32:41.847271 24661 log.cc:824] T 500f83b5e96944ba86d482fc5989e64c P f72f6d4758914628b0e6858beaa29ebf: Log is configured to *not* fsync() on all Append() calls
I20260806 06:32:41.848052 24662 tablet_bootstrap.cc:492] T 500f83b5e96944ba86d482fc5989e64c P bb30337f6b5649df8c3b08d4c616cdca: No bootstrap required, opened a new log
I20260806 06:32:41.848122 24662 ts_tablet_manager.cc:1397] T 500f83b5e96944ba86d482fc5989e64c P bb30337f6b5649df8c3b08d4c616cdca: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.003s
I20260806 06:32:41.850842 24662 raft_consensus.cc:348] T 500f83b5e96944ba86d482fc5989e64c P bb30337f6b5649df8c3b08d4c616cdca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb30337f6b5649df8c3b08d4c616cdca" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 38943 } } peers { permanent_uuid: "f72f6d4758914628b0e6858beaa29ebf" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 41377 } } peers { permanent_uuid: "0e6b240031da49b690c00599ba37b872" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 38285 } }
I20260806 06:32:41.851022 24662 raft_consensus.cc:374] T 500f83b5e96944ba86d482fc5989e64c P bb30337f6b5649df8c3b08d4c616cdca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:32:41.851064 24662 raft_consensus.cc:729] T 500f83b5e96944ba86d482fc5989e64c P bb30337f6b5649df8c3b08d4c616cdca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb30337f6b5649df8c3b08d4c616cdca, State: Initialized, Role: FOLLOWER
I20260806 06:32:41.851213 24662 consensus_queue.cc:260] T 500f83b5e96944ba86d482fc5989e64c P bb30337f6b5649df8c3b08d4c616cdca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb30337f6b5649df8c3b08d4c616cdca" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 38943 } } peers { permanent_uuid: "f72f6d4758914628b0e6858beaa29ebf" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 41377 } } peers { permanent_uuid: "0e6b240031da49b690c00599ba37b872" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 38285 } }
I20260806 06:32:41.851476 24662 ts_tablet_manager.cc:1428] T 500f83b5e96944ba86d482fc5989e64c P bb30337f6b5649df8c3b08d4c616cdca: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:32:41.851610 24210 heartbeater.cc:503] Master 127.7.43.126:33613 was elected leader, sending a full tablet report...
I20260806 06:32:41.852707 24661 tablet_bootstrap.cc:492] T 500f83b5e96944ba86d482fc5989e64c P f72f6d4758914628b0e6858beaa29ebf: No bootstrap required, opened a new log
I20260806 06:32:41.852777 24661 ts_tablet_manager.cc:1397] T 500f83b5e96944ba86d482fc5989e64c P f72f6d4758914628b0e6858beaa29ebf: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.000s
I20260806 06:32:41.854751 24661 raft_consensus.cc:348] T 500f83b5e96944ba86d482fc5989e64c P f72f6d4758914628b0e6858beaa29ebf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb30337f6b5649df8c3b08d4c616cdca" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 38943 } } peers { permanent_uuid: "f72f6d4758914628b0e6858beaa29ebf" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 41377 } } peers { permanent_uuid: "0e6b240031da49b690c00599ba37b872" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 38285 } }
I20260806 06:32:41.854912 24661 raft_consensus.cc:374] T 500f83b5e96944ba86d482fc5989e64c P f72f6d4758914628b0e6858beaa29ebf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:32:41.854959 24661 raft_consensus.cc:729] T 500f83b5e96944ba86d482fc5989e64c P f72f6d4758914628b0e6858beaa29ebf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f72f6d4758914628b0e6858beaa29ebf, State: Initialized, Role: FOLLOWER
I20260806 06:32:41.855105 24661 consensus_queue.cc:260] T 500f83b5e96944ba86d482fc5989e64c P f72f6d4758914628b0e6858beaa29ebf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb30337f6b5649df8c3b08d4c616cdca" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 38943 } } peers { permanent_uuid: "f72f6d4758914628b0e6858beaa29ebf" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 41377 } } peers { permanent_uuid: "0e6b240031da49b690c00599ba37b872" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 38285 } }
I20260806 06:32:41.855373 24661 ts_tablet_manager.cc:1428] T 500f83b5e96944ba86d482fc5989e64c P f72f6d4758914628b0e6858beaa29ebf: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:32:41.855489 24432 heartbeater.cc:503] Master 127.7.43.126:33613 was elected leader, sending a full tablet report...
Aug 06 06:32:41 dist-test-slave-6zbq krb5kdc[23183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997961, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.126@KRBTEST.COM for HTTP/127.7.43.84@KRBTEST.COM
W20260806 06:32:41.938896 24211 tablet.cc:2369] T 500f83b5e96944ba86d482fc5989e64c P bb30337f6b5649df8c3b08d4c616cdca: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 06:32:41.946496 24433 tablet.cc:2369] T 500f83b5e96944ba86d482fc5989e64c P f72f6d4758914628b0e6858beaa29ebf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 06:32:42.004268 24635 tablet.cc:2369] T 500f83b5e96944ba86d482fc5989e64c P 0e6b240031da49b690c00599ba37b872: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 06:32:42 dist-test-slave-6zbq krb5kdc[23183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997961, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.126@KRBTEST.COM for HTTP/127.7.43.84@KRBTEST.COM
I20260806 06:32:42.234727 24660 raft_consensus.cc:482] T 500f83b5e96944ba86d482fc5989e64c P 0e6b240031da49b690c00599ba37b872 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:32:42.234882 24660 raft_consensus.cc:504] T 500f83b5e96944ba86d482fc5989e64c P 0e6b240031da49b690c00599ba37b872 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb30337f6b5649df8c3b08d4c616cdca" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 38943 } } peers { permanent_uuid: "f72f6d4758914628b0e6858beaa29ebf" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 41377 } } peers { permanent_uuid: "0e6b240031da49b690c00599ba37b872" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 38285 } }
I20260806 06:32:42.235319 24660 leader_election.cc:288] T 500f83b5e96944ba86d482fc5989e64c P 0e6b240031da49b690c00599ba37b872 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb30337f6b5649df8c3b08d4c616cdca (127.7.43.65:38943), f72f6d4758914628b0e6858beaa29ebf (127.7.43.66:41377)
I20260806 06:32:42.242488 24637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:32:42.235582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:38943 (local address 127.7.43.67:53291)
0806 06:32:42.235706 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 06:32:42.235716 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:32:42.235758 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:32:42.236034 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:32:42.236036 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:32:42.236037 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:32:42.236187 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:32:42.236191 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:32:42.240063 (+ 3872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:32:42.240065 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:32:42.241324 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:32:42.241327 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:32:42.242213 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:32:42.242214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:32:42.242299 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:32:42.242302 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:32:42.242367 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 06:32:42.242412 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102}
I20260806 06:32:42.243187 24651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:32:42.235840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:53291 (local address 127.7.43.65:38943)
0806 06:32:42.235887 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 06:32:42.235889 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:32:42.235897 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:32:42.235926 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:32:42.235927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:32:42.235936 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:32:42.235984 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:32:42.236384 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:32:42.238457 (+ 2073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:32:42.241432 (+ 2975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:32:42.242082 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:32:42.242153 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:32:42.242166 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0806 06:32:42.243057 (+ 891us) server_negotiation.cc:299] Negotiation successful
0806 06:32:42.243107 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 06:32:42.243448 24150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "500f83b5e96944ba86d482fc5989e64c" candidate_uuid: "0e6b240031da49b690c00599ba37b872" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb30337f6b5649df8c3b08d4c616cdca" is_pre_election: true
I20260806 06:32:42.243602 24150 raft_consensus.cc:2454] T 500f83b5e96944ba86d482fc5989e64c P bb30337f6b5649df8c3b08d4c616cdca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e6b240031da49b690c00599ba37b872 in term 0.
I20260806 06:32:42.243896 24468 leader_election.cc:302] T 500f83b5e96944ba86d482fc5989e64c P 0e6b240031da49b690c00599ba37b872 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e6b240031da49b690c00599ba37b872, bb30337f6b5649df8c3b08d4c616cdca; no voters:
I20260806 06:32:42.244028 24660 raft_consensus.cc:2781] T 500f83b5e96944ba86d482fc5989e64c P 0e6b240031da49b690c00599ba37b872 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:32:42.244066 24660 raft_consensus.cc:482] T 500f83b5e96944ba86d482fc5989e64c P 0e6b240031da49b690c00599ba37b872 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:32:42.244087 24660 raft_consensus.cc:3037] T 500f83b5e96944ba86d482fc5989e64c P 0e6b240031da49b690c00599ba37b872 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:32:42.245090 24660 raft_consensus.cc:504] T 500f83b5e96944ba86d482fc5989e64c P 0e6b240031da49b690c00599ba37b872 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb30337f6b5649df8c3b08d4c616cdca" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 38943 } } peers { permanent_uuid: "f72f6d4758914628b0e6858beaa29ebf" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 41377 } } peers { permanent_uuid: "0e6b240031da49b690c00599ba37b872" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 38285 } }
I20260806 06:32:42.245250 24660 leader_election.cc:288] T 500f83b5e96944ba86d482fc5989e64c P 0e6b240031da49b690c00599ba37b872 [CANDIDATE]: Term 1 election: Requested vote from peers bb30337f6b5649df8c3b08d4c616cdca (127.7.43.65:38943), f72f6d4758914628b0e6858beaa29ebf (127.7.43.66:41377)
I20260806 06:32:42.245675 24150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "500f83b5e96944ba86d482fc5989e64c" candidate_uuid: "0e6b240031da49b690c00599ba37b872" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb30337f6b5649df8c3b08d4c616cdca"
I20260806 06:32:42.245759 24150 raft_consensus.cc:3037] T 500f83b5e96944ba86d482fc5989e64c P bb30337f6b5649df8c3b08d4c616cdca [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:32:42.246739 24150 raft_consensus.cc:2454] T 500f83b5e96944ba86d482fc5989e64c P bb30337f6b5649df8c3b08d4c616cdca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e6b240031da49b690c00599ba37b872 in term 1.
I20260806 06:32:42.246994 24468 leader_election.cc:302] T 500f83b5e96944ba86d482fc5989e64c P 0e6b240031da49b690c00599ba37b872 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e6b240031da49b690c00599ba37b872, bb30337f6b5649df8c3b08d4c616cdca; no voters:
I20260806 06:32:42.247118 24660 raft_consensus.cc:2781] T 500f83b5e96944ba86d482fc5989e64c P 0e6b240031da49b690c00599ba37b872 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:32:42.247279 24660 raft_consensus.cc:686] T 500f83b5e96944ba86d482fc5989e64c P 0e6b240031da49b690c00599ba37b872 [term 1 LEADER]: Becoming Leader. State: Replica: 0e6b240031da49b690c00599ba37b872, State: Running, Role: LEADER
I20260806 06:32:42.247380 24660 consensus_queue.cc:237] T 500f83b5e96944ba86d482fc5989e64c P 0e6b240031da49b690c00599ba37b872 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb30337f6b5649df8c3b08d4c616cdca" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 38943 } } peers { permanent_uuid: "f72f6d4758914628b0e6858beaa29ebf" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 41377 } } peers { permanent_uuid: "0e6b240031da49b690c00599ba37b872" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 38285 } }
I20260806 06:32:42.248232 23916 catalog_manager.cc:5495] T 500f83b5e96944ba86d482fc5989e64c P 0e6b240031da49b690c00599ba37b872 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e6b240031da49b690c00599ba37b872 (127.7.43.67). New cstate: current_term: 1 leader_uuid: "0e6b240031da49b690c00599ba37b872" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb30337f6b5649df8c3b08d4c616cdca" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f72f6d4758914628b0e6858beaa29ebf" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 41377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e6b240031da49b690c00599ba37b872" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 38285 } health_report { overall_health: HEALTHY } } }
I20260806 06:32:42.252785 24637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:32:42.236056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.66:41377 (local address 127.7.43.67:47305)
0806 06:32:42.242562 (+ 6506us) negotiation.cc:107] Waiting for socket to connect
0806 06:32:42.242571 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:32:42.242610 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:32:42.249490 (+ 6880us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:32:42.249491 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:32:42.249493 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:32:42.249536 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:32:42.249538 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:32:42.250497 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:32:42.250498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:32:42.251586 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:32:42.251590 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:32:42.252561 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:32:42.252562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:32:42.252626 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:32:42.252630 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:32:42.252675 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 06:32:42.252714 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6495,"thread_start_us":74,"threads_started":1}
I20260806 06:32:42.253126 24654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:32:42.236327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:47305 (local address 127.7.43.66:41377)
0806 06:32:42.236391 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 06:32:42.236393 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:32:42.242796 (+ 6403us) server_negotiation.cc:407] Connection header received
0806 06:32:42.242827 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:32:42.242828 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:32:42.242839 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:32:42.242879 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:32:42.249630 (+ 6751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:32:42.250401 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:32:42.251691 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:32:42.252439 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:32:42.252513 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:32:42.252525 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 06:32:42.253017 (+ 492us) server_negotiation.cc:299] Negotiation successful
0806 06:32:42.253061 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 06:32:42.255429 24365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "500f83b5e96944ba86d482fc5989e64c" candidate_uuid: "0e6b240031da49b690c00599ba37b872" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f72f6d4758914628b0e6858beaa29ebf" is_pre_election: true
I20260806 06:32:42.255580 24365 raft_consensus.cc:2454] T 500f83b5e96944ba86d482fc5989e64c P f72f6d4758914628b0e6858beaa29ebf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e6b240031da49b690c00599ba37b872 in term 0.
I20260806 06:32:42.255704 24361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "500f83b5e96944ba86d482fc5989e64c" candidate_uuid: "0e6b240031da49b690c00599ba37b872" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f72f6d4758914628b0e6858beaa29ebf"
I20260806 06:32:42.255766 24361 raft_consensus.cc:3037] T 500f83b5e96944ba86d482fc5989e64c P f72f6d4758914628b0e6858beaa29ebf [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:32:42.256641 24361 raft_consensus.cc:2454] T 500f83b5e96944ba86d482fc5989e64c P f72f6d4758914628b0e6858beaa29ebf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e6b240031da49b690c00599ba37b872 in term 1.
I20260806 06:32:42.260712 24657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:32:42.257428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55742 (local address 127.7.43.67:38285)
0806 06:32:42.257464 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 06:32:42.257466 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:32:42.257472 (+ 6us) server_negotiation.cc:407] Connection header received
0806 06:32:42.257501 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:32:42.257502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:32:42.257519 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:32:42.257575 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:32:42.257773 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:32:42.258515 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:32:42.259617 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:32:42.259923 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:32:42.259984 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:32:42.260178 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:32:42.260321 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:32:42.260397 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 06:32:42.260607 (+ 210us) server_negotiation.cc:299] Negotiation successful
0806 06:32:42.260626 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:32:42.271486 24361 raft_consensus.cc:1261] T 500f83b5e96944ba86d482fc5989e64c P f72f6d4758914628b0e6858beaa29ebf [term 1 FOLLOWER]: Refusing update from remote peer 0e6b240031da49b690c00599ba37b872: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:32:42.271908 24673 consensus_queue.cc:1035] T 500f83b5e96944ba86d482fc5989e64c P 0e6b240031da49b690c00599ba37b872 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f72f6d4758914628b0e6858beaa29ebf" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 41377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:32:42.274224 24150 raft_consensus.cc:1261] T 500f83b5e96944ba86d482fc5989e64c P bb30337f6b5649df8c3b08d4c616cdca [term 1 FOLLOWER]: Refusing update from remote peer 0e6b240031da49b690c00599ba37b872: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:32:42.274597 24673 consensus_queue.cc:1035] T 500f83b5e96944ba86d482fc5989e64c P 0e6b240031da49b690c00599ba37b872 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb30337f6b5649df8c3b08d4c616cdca" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:32:42.278453 24680 mvcc.cc:205] Tried to move back new op lower bound from 7315447653460967424 to 7315447653365825536. Current Snapshot: MvccSnapshot[applied={T|T < 7315447653460967424}]
I20260806 06:32:42.279026 24683 mvcc.cc:205] Tried to move back new op lower bound from 7315447653460967424 to 7315447653365825536. Current Snapshot: MvccSnapshot[applied={T|T < 7315447653460967424}]
I20260806 06:32:42.280579 24681 mvcc.cc:205] Tried to move back new op lower bound from 7315447653460967424 to 7315447653365825536. Current Snapshot: MvccSnapshot[applied={T|T < 7315447653460967424}]
I20260806 06:32:42.284965 23916 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50548:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:32:42.285076 23916 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50548:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:32:42.286479 23916 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 62664b2dd10242a6954edf80774433bf: Sending DeleteTablet for 3 replicas of tablet 500f83b5e96944ba86d482fc5989e64c
I20260806 06:32:42.287082 24123 tablet_service.cc:1514] Processing DeleteTablet for tablet 500f83b5e96944ba86d482fc5989e64c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:32:42 UTC) from {username='kudu', principal='kudu/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:53824
I20260806 06:32:42.287361 24321 tablet_service.cc:1514] Processing DeleteTablet for tablet 500f83b5e96944ba86d482fc5989e64c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:32:42 UTC) from {username='kudu', principal='kudu/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:52640
I20260806 06:32:42.287590 24506 tablet_service.cc:1514] Processing DeleteTablet for tablet 500f83b5e96944ba86d482fc5989e64c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:32:42 UTC) from {username='kudu', principal='kudu/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:55740
I20260806 06:32:42.288393 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 24027
I20260806 06:32:42.290788 24691 tablet_replica.cc:330] T 500f83b5e96944ba86d482fc5989e64c P f72f6d4758914628b0e6858beaa29ebf: stopping tablet replica
I20260806 06:32:42.290943 24691 raft_consensus.cc:2229] T 500f83b5e96944ba86d482fc5989e64c P f72f6d4758914628b0e6858beaa29ebf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:32:42.291106 24691 raft_consensus.cc:2258] T 500f83b5e96944ba86d482fc5989e64c P f72f6d4758914628b0e6858beaa29ebf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:32:42.292484 24692 tablet_replica.cc:330] T 500f83b5e96944ba86d482fc5989e64c P 0e6b240031da49b690c00599ba37b872: stopping tablet replica
I20260806 06:32:42.292624 24692 raft_consensus.cc:2229] T 500f83b5e96944ba86d482fc5989e64c P 0e6b240031da49b690c00599ba37b872 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:32:42.292814 24692 raft_consensus.cc:2258] T 500f83b5e96944ba86d482fc5989e64c P 0e6b240031da49b690c00599ba37b872 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:32:42.295506 24691 ts_tablet_manager.cc:1905] T 500f83b5e96944ba86d482fc5989e64c P f72f6d4758914628b0e6858beaa29ebf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:32:42.297799 24691 ts_tablet_manager.cc:1918] T 500f83b5e96944ba86d482fc5989e64c P f72f6d4758914628b0e6858beaa29ebf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:32:42.297892 24691 log.cc:1196] T 500f83b5e96944ba86d482fc5989e64c P f72f6d4758914628b0e6858beaa29ebf: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ts-1/wal/wals/500f83b5e96944ba86d482fc5989e64c
I20260806 06:32:42.298213 24691 ts_tablet_manager.cc:1939] T 500f83b5e96944ba86d482fc5989e64c P f72f6d4758914628b0e6858beaa29ebf: Deleting consensus metadata
I20260806 06:32:42.299623 24692 ts_tablet_manager.cc:1905] T 500f83b5e96944ba86d482fc5989e64c P 0e6b240031da49b690c00599ba37b872: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:32:42.302268 24692 ts_tablet_manager.cc:1918] T 500f83b5e96944ba86d482fc5989e64c P 0e6b240031da49b690c00599ba37b872: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:32:42.302456 24692 log.cc:1196] T 500f83b5e96944ba86d482fc5989e64c P 0e6b240031da49b690c00599ba37b872: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785997861828271-7341-0/minicluster-data/ts-2/wal/wals/500f83b5e96944ba86d482fc5989e64c
I20260806 06:32:42.302800 24692 ts_tablet_manager.cc:1939] T 500f83b5e96944ba86d482fc5989e64c P 0e6b240031da49b690c00599ba37b872: Deleting consensus metadata
I20260806 06:32:42.303844 23891 catalog_manager.cc:4841] TS 0e6b240031da49b690c00599ba37b872 (127.7.43.67:38285): tablet 500f83b5e96944ba86d482fc5989e64c (table test-table [id=13264de7acff40f297f306089048bf0a]) successfully deleted
I20260806 06:32:42.304073 23891 catalog_manager.cc:4841] TS f72f6d4758914628b0e6858beaa29ebf (127.7.43.66:41377): tablet 500f83b5e96944ba86d482fc5989e64c (table test-table [id=13264de7acff40f297f306089048bf0a]) successfully deleted
W20260806 06:32:42.318327 23892 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.43.65:38943 (error 108)
I20260806 06:32:42.322540 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 24217
I20260806 06:32:42.332746 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 24438
I20260806 06:32:42.338992 24694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:32:42.338738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:38943 (local address 127.0.0.1:53830)
0806 06:32:42.338876 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 06:32:42.338928 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.65:38943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":56,"threads_started":1}
W20260806 06:32:42.339258 23892 catalog_manager.cc:4584] TS bb30337f6b5649df8c3b08d4c616cdca (127.7.43.65:38943): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 500f83b5e96944ba86d482fc5989e64c: Network error: Client connection negotiation failed: client connection to 127.7.43.65:38943: connect: Connection refused (error 111)
I20260806 06:32:42.341735 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 23865
I20260806 06:32:42.353785 7341 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 06:32:42.697300 7341 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 06:32:42.697404 7341 mini_ranger.cc:66] Stopping Ranger...
I20260806 06:32:43.040939 7341 mini_ranger.cc:68] Stopped Ranger
2026-08-06 06:32:43.041 UTC [23230] LOG: received smart shutdown request
2026-08-06 06:32:43.044 UTC [23230] LOG: background worker "logical replication launcher" (PID 23251) exited with exit code 1
2026-08-06 06:32:43.044 UTC [23238] LOG: shutting down
2026-08-06 06:32:43.175 UTC [23230] LOG: database system is shut down
2026-08-06T06:32:43Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (45448 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:32:43 dist-test-slave-6zbq krb5kdc[24709](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:32:43 dist-test-slave-6zbq krb5kdc[24709](info): set up 2 sockets
Aug 06 06:32:43 dist-test-slave-6zbq krb5kdc[24709](info): commencing operation
krb5kdc: starting...
W20260806 06:32:44.375401 7341 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Aug 06 06:32:44 dist-test-slave-6zbq krb5kdc[24709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997964, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:32:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:32:44Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.7.43.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.7.43.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.7.43.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/rangeradmin_127.7.43.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.7.43.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/rangeradmin_127.7.43.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.7.43.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.7.43.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.7.43.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/rangerlookup_127.7.43.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.7.43.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/rangerlookup_127.7.43.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.43.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.7.43.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.43.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/HTTP_127.7.43.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.43.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/HTTP_127.7.43.84@KRBTEST.COM.keytab.
I20260806 06:32:44.518781 7341 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskfvM9Wt/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/postgres -l logfile start
2026-08-06 06:32:46.652 UTC [24753] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 06:32:46.652 UTC [24753] LOG: listening on IPv4 address "127.7.43.84", port 36109
2026-08-06 06:32:46.655 UTC [24753] LOG: listening on Unix socket "/tmp/.s.PGSQL.36109"
2026-08-06 06:32:46.674 UTC [24756] LOG: database system was shut down at 2026-08-06 06:32:45 UTC
2026-08-06 06:32:46.677 UTC [24753] LOG: database system is ready to accept connections
I20260806 06:32:47.706118 7341 mini_postgres.cc:96] Postgres bound to 36109
2026-08-06 06:32:47.712 UTC [24767] FATAL: database "slave" does not exist
127.7.43.84:36109 - accepting connections
I20260806 06:32:47.713137 7341 mini_ranger.cc:161] Starting Ranger...
I20260806 06:32:47.726945 7341 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 06:32:48.100720 7341 mini_ranger.cc:87] Created ranger Postgres database
I20260806 06:32:48.100790 7341 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ranger-admin
2026-08-06 06:32:48,374 [I] DB FLAVOR :POSTGRES
2026-08-06 06:32:48,374 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 06:32:48,374 [I] Checking connection..
2026-08-06 06:32:48,374 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvM9Wt/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvM9Wt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.43.84:36109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 06:32:48,635 [I] Checking connection passed.
2026-08-06 06:32:48,635 [I] --------- Verifying version history table ---------
2026-08-06 06:32:48,635 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvM9Wt/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvM9Wt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.43.84:36109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 06:32:48,885 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 06:32:48,886 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvM9Wt/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvM9Wt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.43.84:36109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 06:32:49,112 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 06:32:49,112 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 06:32:49,113 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvM9Wt/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvM9Wt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.43.84:36109/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskfvM9Wt/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 06:32:49.347 UTC [24852] WARNING: there is no transaction in progress
2026-08-06 06:32:49,354 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 06:32:49,355 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvM9Wt/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvM9Wt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.43.84:36109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 06:32:49,596 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 06:32:49,596 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 06:32:49,596 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvM9Wt/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvM9Wt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.43.84:36109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 06:32:49,842 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvM9Wt/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvM9Wt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.43.84:36109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 06:32:50,085 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvM9Wt/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvM9Wt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.43.84:36109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-6zbq.c.gcp-upstream.internal','N') ;"
2026-08-06 06:32:50,319 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvM9Wt/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvM9Wt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.43.84:36109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 06:32:50,559 [I] Table x_portal_user does not exist in database ranger
2026-08-06 06:32:50,559 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvM9Wt/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvM9Wt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.43.84:36109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 06:32:50,800 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 06:32:50,801 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvM9Wt/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvM9Wt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.43.84:36109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 06:32:51,036 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvM9Wt/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvM9Wt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.43.84:36109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 06:32:51,291 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 06:32:51,291 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvM9Wt/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvM9Wt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.43.84:36109/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskfvM9Wt/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 06:32:52.134 UTC [25010] WARNING: there is no transaction in progress
2026-08-06 06:32:52.150 UTC [25010] WARNING: there is no transaction in progress
2026-08-06 06:32:52.164 UTC [25010] WARNING: there is no transaction in progress
2026-08-06 06:32:52.178 UTC [25010] WARNING: there is no transaction in progress
2026-08-06 06:32:52.194 UTC [25010] WARNING: there is no transaction in progress
2026-08-06 06:32:52.213 UTC [25010] WARNING: there is no transaction in progress
2026-08-06 06:32:52.272 UTC [25010] WARNING: there is no transaction in progress
2026-08-06 06:32:52.278 UTC [25010] WARNING: there is no transaction in progress
2026-08-06 06:32:52.290 UTC [25010] WARNING: there is no transaction in progress
2026-08-06 06:32:52.299 UTC [25010] WARNING: there is no transaction in progress
2026-08-06 06:32:52.308 UTC [25010] WARNING: there is no transaction in progress
2026-08-06 06:32:52.321 UTC [25010] WARNING: there is no transaction in progress
2026-08-06 06:32:52.329 UTC [25010] WARNING: there is no transaction in progress
2026-08-06 06:32:52.335 UTC [25010] WARNING: there is no transaction in progress
2026-08-06 06:32:52.340 UTC [25010] WARNING: there is no transaction in progress
2026-08-06 06:32:52.346 UTC [25010] WARNING: there is no transaction in progress
2026-08-06 06:32:52,901 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 06:32:52,901 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvM9Wt/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvM9Wt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.43.84:36109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 06:32:53,126 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 06:32:53,127 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvM9Wt/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvM9Wt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.43.84:36109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 06:32:53,377 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 06:32:53,378 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvM9Wt/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvM9Wt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.43.84:36109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 06:32:53,618 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 06:32:53,618 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvM9Wt/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvM9Wt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.43.84:36109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 06:32:53,845 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvM9Wt/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvM9Wt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.43.84:36109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 06:32:54,093 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvM9Wt/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvM9Wt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.43.84:36109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-6zbq.c.gcp-upstream.internal';"
2026-08-06 06:32:54,329 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 06:32:54,330 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvM9Wt/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvM9Wt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.43.84:36109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 06:32:54,560 [I] DB_PATCHES have already been applied
I20260806 06:32:54.565418 7341 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ranger-admin:/tmp/dist-test-taskfvM9Wt/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskfvM9Wt/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskfvM9Wt/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvM9Wt/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 06:32:54.565475 7341 mini_ranger.cc:193] Using host: 127.7.43.84
I20260806 06:32:54.568965 7341 mini_ranger.cc:227] Ranger admin URL: http://127.7.43.84:51409
Aug 06, 2026 6:32:54 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 6:32:54 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 6:32:54 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 6:32:54 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 6:32:54 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 6:32:55 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 6:32:55 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 06:32:55.723783 7341 mini_ranger.cc:160] Time spent starting Ranger: real 8.011s user 0.000s sys 0.009s
Aug 06, 2026 6:33:15 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 6:33:15 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 06, 2026 6:33:15 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 6:33:15 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 6:33:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 6:33:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 6:33:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 6:33:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 6:33:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 6:33:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 6:33:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 6:33:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 6:33:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 6:33:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 6:33:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 6:33:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 6:33:15 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 6:33:16 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 06:33:16.734108 7341 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.7.43.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.7.43.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.7.43.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/rangerkms_127.7.43.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.7.43.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/rangerkms_127.7.43.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.43.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.7.43.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.43.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/HTTP_127.7.43.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.43.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/HTTP_127.7.43.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 06:33:16 dist-test-slave-6zbq krb5kdc[24709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997996, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 06:33:16.822029 7341 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 06:33:16.831921 7341 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 06:33:17.524302 7341 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 06:33:17.524389 7341 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ranger-kms
I20260806 06:33:17.524407 7341 mini_ranger_kms.cc:227] Using postgres at 127.7.43.84:36109
2026-08-06 06:33:17,688 [I] DB FLAVOR :POSTGRES
2026-08-06 06:33:17,689 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 06:33:17,689 [I] Checking connection
2026-08-06 06:33:17,689 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvM9Wt/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.43.84:36109/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 06:33:17,908 [I] connection success
2026-08-06 06:33:17,909 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 06:33:17,909 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 06:33:17,909 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvM9Wt/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.43.84:36109/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 06:33:18,155 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 06:33:18,155 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 06:33:18,155 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 06:33:18,155 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvM9Wt/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.43.84:36109/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 06:33:18,433 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 06:33:18.855939 7341 mini_ranger_kms.cc:310] Created kms service
I20260806 06:33:19.052558 7341 mini_ranger_kms.cc:326] Created kudu user
I20260806 06:33:19.093400 7341 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 06:33:19 dist-test-slave-6zbq krb5kdc[24709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785997996, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.7.43.84@KRBTEST.COM
I20260806 06:33:19.476496 7341 mini_ranger_kms.cc:384] Added ranger policy
I20260806 06:33:19.476608 7341 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ranger-kms:/tmp/dist-test-taskfvM9Wt/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvM9Wt/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskfvM9Wt/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 06:33:19.476629 7341 mini_ranger_kms.cc:240] Using host: 127.7.43.84
I20260806 06:33:19.480397 7341 mini_ranger_kms.cc:276] Ranger KMS PID: 25273
I20260806 06:33:19.480468 7341 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.7.43.84:48483
Aug 06, 2026 6:33:20 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 6:33:20 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 6:33:20 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 6:33:20 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 6:33:20 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 6:33:20 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 06 06:33:21 dist-test-slave-6zbq krb5kdc[24709](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785998001, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.43.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:33:21.896857 7341 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.075s user 0.004s sys 0.144s
I20260806 06:33:21.896955 7341 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 06:33:21.896997 7341 mini_ranger_kms.cc:401] 127.7.43.84:48483/kms/v1/keys
Aug 06 06:33:21 dist-test-slave-6zbq krb5kdc[24709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998001, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.43.84@KRBTEST.COM for HTTP/127.7.43.84@KRBTEST.COM
Aug 06 06:33:22 dist-test-slave-6zbq krb5kdc[24709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998001, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.43.84@KRBTEST.COM for HTTP/127.7.43.84@KRBTEST.COM
Aug 06 06:33:24 dist-test-slave-6zbq krb5kdc[24709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.7.43.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
I20260806 06:33:24.545249 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:43181
--webserver_interface=127.7.43.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:44991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.43.126:43181
--ranger_config_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.43.84:48483/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:33:24.629638 25339 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:33:24.629828 25339 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:33:24.629848 25339 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:33:24.630960 25339 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:33:24.630998 25339 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:33:24.631011 25339 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:33:24.631023 25339 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:33:24.631033 25339 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:33:24.632539 25339 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:44991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/master-0/wal
--ranger_kms_url=127.7.43.84:48483/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.43.126:43181
--ranger_config_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:43181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.43.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25339
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:33:24.632747 25339 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:33:24.632954 25339 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:33:24.637701 25345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:33:24.637821 25347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:33:24.637810 25339 server_base.cc:1034] running on GCE node
W20260806 06:33:24.637727 25344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:33:24.638126 25339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:33:24.638451 25339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:33:24.639587 25339 hybrid_clock.cc:648] HybridClock initialized: now 1785998004639575 us; error 31 us; skew 500 ppm
Aug 06 06:33:24 dist-test-slave-6zbq krb5kdc[24709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998004, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:33:24.644534 25339 init.cc:377] Logged in from keytab as kudu/127.7.43.126@KRBTEST.COM (short username kudu)
I20260806 06:33:24.644999 25339 webserver.cc:460] Webserver started at http://127.7.43.126:32771/ using document root <none> and password file <none>
I20260806 06:33:24.645220 25339 fs_manager.cc:359] Metadata directory not provided
I20260806 06:33:24.645260 25339 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:33:24.645366 25339 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 06:33:24 dist-test-slave-6zbq krb5kdc[24709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998004, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.126@KRBTEST.COM for HTTP/127.7.43.84@KRBTEST.COM
I20260806 06:33:24.703078 25339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/master-0/data/instance:
uuid: "901f3fa5cdb240308ecdd5c870a2b433"
format_stamp: "Formatted at 2026-08-06 06:33:24 on dist-test-slave-6zbq"
server_key: "711b730f8eef114765a53e7f2802d255"
server_key_iv: "5c42c1310e6fbbfa6c15809f67139e58"
server_key_version: "kuduclusterkey@0"
I20260806 06:33:24.703450 25339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/master-0/wal/instance:
uuid: "901f3fa5cdb240308ecdd5c870a2b433"
format_stamp: "Formatted at 2026-08-06 06:33:24 on dist-test-slave-6zbq"
server_key: "711b730f8eef114765a53e7f2802d255"
server_key_iv: "5c42c1310e6fbbfa6c15809f67139e58"
server_key_version: "kuduclusterkey@0"
I20260806 06:33:24.704843 25339 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260806 06:33:24.718469 25360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:24.718640 25339 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 06:33:24.718748 25339 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "901f3fa5cdb240308ecdd5c870a2b433"
format_stamp: "Formatted at 2026-08-06 06:33:24 on dist-test-slave-6zbq"
server_key: "711b730f8eef114765a53e7f2802d255"
server_key_iv: "5c42c1310e6fbbfa6c15809f67139e58"
server_key_version: "kuduclusterkey@0"
I20260806 06:33:24.718822 25339 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:33:24.735952 25339 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:33:24.737644 25339 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:33:24.737783 25339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:33:24.742224 25339 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.126:43181
I20260806 06:33:24.742256 25439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.126:43181 every 8 connection(s)
I20260806 06:33:24.742614 25339 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:33:24.743377 25440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:33:24.746434 25440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 901f3fa5cdb240308ecdd5c870a2b433: Bootstrap starting.
I20260806 06:33:24.747138 25440 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 901f3fa5cdb240308ecdd5c870a2b433: Neither blocks nor log segments found. Creating new log.
I20260806 06:33:24.747449 25440 log.cc:824] T 00000000000000000000000000000000 P 901f3fa5cdb240308ecdd5c870a2b433: Log is configured to *not* fsync() on all Append() calls
I20260806 06:33:24.748189 25440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 901f3fa5cdb240308ecdd5c870a2b433: No bootstrap required, opened a new log
I20260806 06:33:24.750471 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 25339
I20260806 06:33:24.750563 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/master-0/wal/instance
I20260806 06:33:24.750733 7341 external_mini_cluster.cc:1428] Setting key 5b315925a4c53b6d4f8f14550228f87f
I20260806 06:33:24.750977 25440 raft_consensus.cc:348] T 00000000000000000000000000000000 P 901f3fa5cdb240308ecdd5c870a2b433 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "901f3fa5cdb240308ecdd5c870a2b433" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 43181 } }
I20260806 06:33:24.751405 25440 raft_consensus.cc:374] T 00000000000000000000000000000000 P 901f3fa5cdb240308ecdd5c870a2b433 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:33:24.751464 25440 raft_consensus.cc:729] T 00000000000000000000000000000000 P 901f3fa5cdb240308ecdd5c870a2b433 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 901f3fa5cdb240308ecdd5c870a2b433, State: Initialized, Role: FOLLOWER
I20260806 06:33:24.751598 25440 consensus_queue.cc:260] T 00000000000000000000000000000000 P 901f3fa5cdb240308ecdd5c870a2b433 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "901f3fa5cdb240308ecdd5c870a2b433" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 43181 } }
I20260806 06:33:24.751690 25440 raft_consensus.cc:388] T 00000000000000000000000000000000 P 901f3fa5cdb240308ecdd5c870a2b433 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:33:24.751718 25440 raft_consensus.cc:482] T 00000000000000000000000000000000 P 901f3fa5cdb240308ecdd5c870a2b433 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:33:24.751783 25440 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 901f3fa5cdb240308ecdd5c870a2b433 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:33:24.752441 25440 raft_consensus.cc:504] T 00000000000000000000000000000000 P 901f3fa5cdb240308ecdd5c870a2b433 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "901f3fa5cdb240308ecdd5c870a2b433" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 43181 } }
I20260806 06:33:24.752550 25440 leader_election.cc:302] T 00000000000000000000000000000000 P 901f3fa5cdb240308ecdd5c870a2b433 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 901f3fa5cdb240308ecdd5c870a2b433; no voters:
I20260806 06:33:24.752736 25440 leader_election.cc:288] T 00000000000000000000000000000000 P 901f3fa5cdb240308ecdd5c870a2b433 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:33:24.752892 25447 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 901f3fa5cdb240308ecdd5c870a2b433 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:33:24.752977 25440 sys_catalog.cc:564] T 00000000000000000000000000000000 P 901f3fa5cdb240308ecdd5c870a2b433 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:33:24.753062 25447 raft_consensus.cc:686] T 00000000000000000000000000000000 P 901f3fa5cdb240308ecdd5c870a2b433 [term 1 LEADER]: Becoming Leader. State: Replica: 901f3fa5cdb240308ecdd5c870a2b433, State: Running, Role: LEADER
I20260806 06:33:24.753214 25447 consensus_queue.cc:237] T 00000000000000000000000000000000 P 901f3fa5cdb240308ecdd5c870a2b433 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "901f3fa5cdb240308ecdd5c870a2b433" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 43181 } }
I20260806 06:33:24.753556 25447 sys_catalog.cc:455] T 00000000000000000000000000000000 P 901f3fa5cdb240308ecdd5c870a2b433 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "901f3fa5cdb240308ecdd5c870a2b433" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "901f3fa5cdb240308ecdd5c870a2b433" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 43181 } } }
I20260806 06:33:24.753671 25447 sys_catalog.cc:458] T 00000000000000000000000000000000 P 901f3fa5cdb240308ecdd5c870a2b433 [sys.catalog]: This master's current role is: LEADER
I20260806 06:33:24.753903 25446 sys_catalog.cc:455] T 00000000000000000000000000000000 P 901f3fa5cdb240308ecdd5c870a2b433 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 901f3fa5cdb240308ecdd5c870a2b433. Latest consensus state: current_term: 1 leader_uuid: "901f3fa5cdb240308ecdd5c870a2b433" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "901f3fa5cdb240308ecdd5c870a2b433" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 43181 } } }
I20260806 06:33:24.754037 25446 sys_catalog.cc:458] T 00000000000000000000000000000000 P 901f3fa5cdb240308ecdd5c870a2b433 [sys.catalog]: This master's current role is: LEADER
I20260806 06:33:24.754442 25440 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 06:33:24.755044 25457 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:33:24.755594 25457 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 06:33:24 dist-test-slave-6zbq krb5kdc[24709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:33:24.762961 25457 catalog_manager.cc:1347] Generated new cluster ID: efe51c35a301468da7d0a86b50f19c88
I20260806 06:33:24.763017 25457 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:33:24.771703 25445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:24.751203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46566 (local address 127.7.43.126:43181)
0806 06:33:24.751365 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:24.751374 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:24.751410 (+ 36us) server_negotiation.cc:407] Connection header received
0806 06:33:24.751523 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:24.751532 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:24.751589 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:24.751737 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:33:24.752440 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:24.753771 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:24.754759 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:24.755413 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:24.755537 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:24.758570 (+ 3033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:33:24.758624 (+ 54us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:33:24.758625 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:33:24.758690 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:33:24.762329 (+ 3639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:24.762729 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:24.762732 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:24.762732 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:24.762845 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:24.770447 (+ 7602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:24.770449 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:24.770449 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:24.770776 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:33:24.770907 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:24.771330 (+ 423us) server_negotiation.cc:299] Negotiation successful
0806 06:33:24.771473 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":76,"threads_started":1}
I20260806 06:33:24.775592 25457 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:33:24.776324 25457 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:33:24.780296 25457 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 901f3fa5cdb240308ecdd5c870a2b433: Generated new TSK 0
I20260806 06:33:24.780505 25457 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.43.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab.
Entry for principal kudu/127.7.43.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab.
I20260806 06:33:25.453851 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.43.84:48483/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.65:0
--local_ip_for_outbound_sockets=127.7.43.65
--webserver_interface=127.7.43.65
--webserver_port=0
--tserver_master_addrs=127.7.43.126:43181
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab
--principal=kudu/127.7.43.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:44991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:33:25.557889 25500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:33:25.558068 25500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:33:25.558089 25500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:33:25.559433 25500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:33:25.559473 25500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:33:25.559499 25500 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.65
I20260806 06:33:25.560931 25500 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:44991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.7.43.84:48483/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab
--principal=kudu/127.7.43.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.43.65
--webserver_port=0
--tserver_master_addrs=127.7.43.126:43181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25500
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.65
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:33:25.561149 25500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:33:25.561364 25500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:33:25.566015 25507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:33:25.566509 25508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:33:25.566751 25511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:33:25.567467 25500 server_base.cc:1034] running on GCE node
I20260806 06:33:25.567670 25500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:33:25.567891 25500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:33:25.569041 25500 hybrid_clock.cc:648] HybridClock initialized: now 1785998005569019 us; error 39 us; skew 500 ppm
Aug 06 06:33:25 dist-test-slave-6zbq krb5kdc[24709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998005, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:33:25.571965 25500 init.cc:377] Logged in from keytab as kudu/127.7.43.65@KRBTEST.COM (short username kudu)
I20260806 06:33:25.572482 25500 webserver.cc:460] Webserver started at http://127.7.43.65:39097/ using document root <none> and password file <none>
I20260806 06:33:25.572784 25500 fs_manager.cc:359] Metadata directory not provided
I20260806 06:33:25.572840 25500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:33:25.572963 25500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 06:33:25 dist-test-slave-6zbq krb5kdc[24709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998005, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.65@KRBTEST.COM for HTTP/127.7.43.84@KRBTEST.COM
I20260806 06:33:25.585330 25500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-0/data/instance:
uuid: "62ff3045c51d4c7aacdff1226a654dc2"
format_stamp: "Formatted at 2026-08-06 06:33:25 on dist-test-slave-6zbq"
server_key: "ae3c9e8e2c8fe1285bb0d9e2cbb344e7"
server_key_iv: "47dc7f7c8b4b3df31f2e53e2c8da5d06"
server_key_version: "kuduclusterkey@0"
I20260806 06:33:25.585796 25500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-0/wal/instance:
uuid: "62ff3045c51d4c7aacdff1226a654dc2"
format_stamp: "Formatted at 2026-08-06 06:33:25 on dist-test-slave-6zbq"
server_key: "ae3c9e8e2c8fe1285bb0d9e2cbb344e7"
server_key_iv: "47dc7f7c8b4b3df31f2e53e2c8da5d06"
server_key_version: "kuduclusterkey@0"
I20260806 06:33:25.587356 25500 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:33:25.602486 25526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:25.602685 25500 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260806 06:33:25.602792 25500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-0/wal
uuid: "62ff3045c51d4c7aacdff1226a654dc2"
format_stamp: "Formatted at 2026-08-06 06:33:25 on dist-test-slave-6zbq"
server_key: "ae3c9e8e2c8fe1285bb0d9e2cbb344e7"
server_key_iv: "47dc7f7c8b4b3df31f2e53e2c8da5d06"
server_key_version: "kuduclusterkey@0"
I20260806 06:33:25.602880 25500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:33:25.614831 25500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:33:25.617069 25500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:33:25.617198 25500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:33:25.617399 25500 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:33:25.617744 25500 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:33:25.617803 25500 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:25.617831 25500 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:33:25.617883 25500 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:25.625322 25500 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.65:36967
I20260806 06:33:25.625358 25683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.65:36967 every 8 connection(s)
I20260806 06:33:25.625720 25500 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
I20260806 06:33:25.629454 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 25500
I20260806 06:33:25.629565 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-0/wal/instance
I20260806 06:33:25.629771 7341 external_mini_cluster.cc:1428] Setting key 8416b4a406a5cb02719af3c8e1996ecd
Aug 06 06:33:25 dist-test-slave-6zbq krb5kdc[24709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998005, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.65@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:33:25.635836 25687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:25.626428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:43181 (local address 127.7.43.65:55519)
0806 06:33:25.626614 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:25.626661 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 06:33:25.627282 (+ 621us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:33:25.627676 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:33:25.627679 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:33:25.627848 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:33:25.628043 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:25.628048 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:25.628730 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:25.628731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:25.629563 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:25.629567 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:25.630093 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:25.630095 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:25.630206 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:25.630649 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:33:25.630659 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:33:25.632189 (+ 1530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:33:25.634197 (+ 2008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:33:25.634204 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:33:25.634206 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:33:25.634518 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:33:25.634747 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:33:25.634748 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:33:25.634748 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:33:25.634874 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:33:25.635194 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:33:25.635201 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:33:25.635366 (+ 165us) client_negotiation.cc:769] Sending connection context
0806 06:33:25.635462 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 06:33:25.635614 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":96,"threads_started":1}
I20260806 06:33:25.636312 25689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:25.626478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:55519 (local address 127.7.43.126:43181)
0806 06:33:25.627466 (+ 988us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:25.627469 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:25.627477 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:33:25.627502 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:25.627502 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:25.627515 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:25.627587 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:33:25.628144 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:25.628648 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:25.629664 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:25.629895 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:25.629964 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:25.632300 (+ 2336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:33:25.632302 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:33:25.632302 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:33:25.632310 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:33:25.634087 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:25.634606 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:25.634608 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:25.634608 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:25.634675 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:25.634946 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:25.634947 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:25.634948 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:25.635089 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:33:25.635164 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:25.636117 (+ 953us) server_negotiation.cc:299] Negotiation successful
0806 06:33:25.636207 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":956,"thread_start_us":81,"threads_started":1}
I20260806 06:33:25.636709 25684 heartbeater.cc:347] Connected to a master server at 127.7.43.126:43181
I20260806 06:33:25.636842 25684 heartbeater.cc:464] Registering TS with master...
I20260806 06:33:25.637060 25684 heartbeater.cc:511] Master 127.7.43.126:43181 requested a full tablet report, sending...
I20260806 06:33:25.637712 25387 ts_manager.cc:194] Registered new tserver with Master: 62ff3045c51d4c7aacdff1226a654dc2 (127.7.43.65:36967)
I20260806 06:33:25.638568 25387 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.65@KRBTEST.COM'} at 127.7.43.65:55519
WARNING: no policy specified for kudu/127.7.43.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab.
Entry for principal kudu/127.7.43.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab.
I20260806 06:33:25.659225 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.43.84:48483/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.66:0
--local_ip_for_outbound_sockets=127.7.43.66
--webserver_interface=127.7.43.66
--webserver_port=0
--tserver_master_addrs=127.7.43.126:43181
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab
--principal=kudu/127.7.43.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:44991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:33:25.752775 25692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:33:25.752957 25692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:33:25.752986 25692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:33:25.754372 25692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:33:25.754416 25692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:33:25.754451 25692 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.66
I20260806 06:33:25.756029 25692 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:44991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.7.43.84:48483/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab
--principal=kudu/127.7.43.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.43.66
--webserver_port=0
--tserver_master_addrs=127.7.43.126:43181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25692
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.66
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:33:25.756258 25692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:33:25.756475 25692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:33:25.761286 25705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:33:25.761333 25702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:33:25.761525 25692 server_base.cc:1034] running on GCE node
W20260806 06:33:25.761714 25701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:33:25.761905 25692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:33:25.762121 25692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
Aug 06 06:33:25 dist-test-slave-6zbq krb5kdc[24709](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785998005, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:33:25.763280 25692 hybrid_clock.cc:648] HybridClock initialized: now 1785998005763258 us; error 36 us; skew 500 ppm
Aug 06 06:33:25 dist-test-slave-6zbq krb5kdc[24709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998005, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:33:25.765511 25692 init.cc:377] Logged in from keytab as kudu/127.7.43.66@KRBTEST.COM (short username kudu)
I20260806 06:33:25.765883 25692 webserver.cc:460] Webserver started at http://127.7.43.66:34803/ using document root <none> and password file <none>
I20260806 06:33:25.766088 25692 fs_manager.cc:359] Metadata directory not provided
I20260806 06:33:25.766134 25692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:33:25.766258 25692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 06:33:25 dist-test-slave-6zbq krb5kdc[24709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998005, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.66@KRBTEST.COM for HTTP/127.7.43.84@KRBTEST.COM
I20260806 06:33:25.780809 25692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-1/data/instance:
uuid: "49a8501de59748449a52600db82ef49f"
format_stamp: "Formatted at 2026-08-06 06:33:25 on dist-test-slave-6zbq"
server_key: "bc5b91b08c153981957f8b4d6d4efc84"
server_key_iv: "cae249eb985f8de762a4d83469903389"
server_key_version: "kuduclusterkey@0"
I20260806 06:33:25.781255 25692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-1/wal/instance:
uuid: "49a8501de59748449a52600db82ef49f"
format_stamp: "Formatted at 2026-08-06 06:33:25 on dist-test-slave-6zbq"
server_key: "bc5b91b08c153981957f8b4d6d4efc84"
server_key_iv: "cae249eb985f8de762a4d83469903389"
server_key_version: "kuduclusterkey@0"
I20260806 06:33:25.787410 25692 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.003s
I20260806 06:33:25.802729 25715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:25.805487 25692 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.001s
I20260806 06:33:25.805578 25692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-1/wal
uuid: "49a8501de59748449a52600db82ef49f"
format_stamp: "Formatted at 2026-08-06 06:33:25 on dist-test-slave-6zbq"
server_key: "bc5b91b08c153981957f8b4d6d4efc84"
server_key_iv: "cae249eb985f8de762a4d83469903389"
server_key_version: "kuduclusterkey@0"
I20260806 06:33:25.805650 25692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:33:25.826876 25692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:33:25.828909 25692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:33:25.829008 25692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:33:25.829205 25692 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:33:25.829533 25692 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:33:25.829577 25692 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:25.829598 25692 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:33:25.829622 25692 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:25.840617 25692 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.66:36295
I20260806 06:33:25.840801 25894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.66:36295 every 8 connection(s)
I20260806 06:33:25.841105 25692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
I20260806 06:33:25.845832 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 25692
I20260806 06:33:25.845923 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-1/wal/instance
I20260806 06:33:25.846086 7341 external_mini_cluster.cc:1428] Setting key 9671bb9aa63f13abbf55a1674764d6ae
Aug 06 06:33:25 dist-test-slave-6zbq krb5kdc[24709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998005, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.66@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:33:25.855707 25689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:25.841942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:54521 (local address 127.7.43.126:43181)
0806 06:33:25.841992 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:25.841994 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:25.842332 (+ 338us) server_negotiation.cc:407] Connection header received
0806 06:33:25.843378 (+ 1046us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:25.843380 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:25.843393 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:25.843424 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:33:25.844417 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:25.845224 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:25.849053 (+ 3829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:25.849361 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:25.849410 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:25.852210 (+ 2800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:33:25.852212 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:33:25.852212 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:33:25.852220 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:33:25.853936 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:25.854473 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:25.854475 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:25.854475 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:25.854543 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:25.854859 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:25.854860 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:25.854861 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:25.855000 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:33:25.855053 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:25.855518 (+ 465us) server_negotiation.cc:299] Negotiation successful
0806 06:33:25.855609 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:33:25.856236 25897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:25.841943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:43181 (local address 127.7.43.66:54521)
0806 06:33:25.842174 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:25.842226 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 06:33:25.843247 (+ 1021us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:33:25.843594 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:33:25.843598 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:33:25.843920 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:33:25.844298 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:25.844305 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:25.845338 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:25.845339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:25.848940 (+ 3601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:25.848944 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:25.849517 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:25.849518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:25.849659 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:25.850273 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:33:25.850283 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:33:25.852086 (+ 1803us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:33:25.854036 (+ 1950us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:33:25.854043 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:33:25.854045 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:33:25.854374 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:33:25.854625 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:33:25.854626 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:33:25.854626 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:33:25.854772 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:33:25.855087 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:33:25.855105 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:33:25.855342 (+ 237us) client_negotiation.cc:769] Sending connection context
0806 06:33:25.855797 (+ 455us) client_negotiation.cc:240] Negotiation successful
0806 06:33:25.855950 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":106,"threads_started":1}
I20260806 06:33:25.856832 25895 heartbeater.cc:347] Connected to a master server at 127.7.43.126:43181
I20260806 06:33:25.856931 25895 heartbeater.cc:464] Registering TS with master...
I20260806 06:33:25.857108 25895 heartbeater.cc:511] Master 127.7.43.126:43181 requested a full tablet report, sending...
I20260806 06:33:25.857502 25387 ts_manager.cc:194] Registered new tserver with Master: 49a8501de59748449a52600db82ef49f (127.7.43.66:36295)
I20260806 06:33:25.857985 25387 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.66@KRBTEST.COM'} at 127.7.43.66:54521
WARNING: no policy specified for kudu/127.7.43.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab.
Entry for principal kudu/127.7.43.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab.
I20260806 06:33:25.876788 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.43.84:48483/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.67:0
--local_ip_for_outbound_sockets=127.7.43.67
--webserver_interface=127.7.43.67
--webserver_port=0
--tserver_master_addrs=127.7.43.126:43181
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab
--principal=kudu/127.7.43.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:44991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:33:25.976445 25900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:33:25.976642 25900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:33:25.976675 25900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:33:25.977802 25900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:33:25.977844 25900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:33:25.977878 25900 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.67
I20260806 06:33:25.979558 25900 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:44991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.7.43.84:48483/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab
--principal=kudu/127.7.43.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.43.67
--webserver_port=0
--tserver_master_addrs=127.7.43.126:43181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25900
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.67
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:33:25.979789 25900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:33:25.980050 25900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:33:25.985169 25908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:33:25.985245 25911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:33:25.985406 25909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:33:25.985816 25900 server_base.cc:1034] running on GCE node
I20260806 06:33:25.986013 25900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:33:25.986238 25900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:33:25.987381 25900 hybrid_clock.cc:648] HybridClock initialized: now 1785998005987370 us; error 28 us; skew 500 ppm
Aug 06 06:33:25 dist-test-slave-6zbq krb5kdc[24709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998005, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:33:25.989864 25900 init.cc:377] Logged in from keytab as kudu/127.7.43.67@KRBTEST.COM (short username kudu)
I20260806 06:33:25.990279 25900 webserver.cc:460] Webserver started at http://127.7.43.67:42371/ using document root <none> and password file <none>
I20260806 06:33:25.990566 25900 fs_manager.cc:359] Metadata directory not provided
I20260806 06:33:25.990621 25900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:33:25.990783 25900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 06:33:25 dist-test-slave-6zbq krb5kdc[24709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998005, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.67@KRBTEST.COM for HTTP/127.7.43.84@KRBTEST.COM
I20260806 06:33:26.012355 25900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-2/data/instance:
uuid: "e10d1db4de26494c81b6bed2e25a2a50"
format_stamp: "Formatted at 2026-08-06 06:33:26 on dist-test-slave-6zbq"
server_key: "dc1b8d6f674cb07a5d435ed4a0bbb462"
server_key_iv: "e3343dc1f6ffd05ba583284de3846362"
server_key_version: "kuduclusterkey@0"
I20260806 06:33:26.012773 25900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-2/wal/instance:
uuid: "e10d1db4de26494c81b6bed2e25a2a50"
format_stamp: "Formatted at 2026-08-06 06:33:26 on dist-test-slave-6zbq"
server_key: "dc1b8d6f674cb07a5d435ed4a0bbb462"
server_key_iv: "e3343dc1f6ffd05ba583284de3846362"
server_key_version: "kuduclusterkey@0"
I20260806 06:33:26.020624 25900 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.001s sys 0.001s
I20260806 06:33:26.039810 25924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:26.040002 25900 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.000s
I20260806 06:33:26.040098 25900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-2/wal
uuid: "e10d1db4de26494c81b6bed2e25a2a50"
format_stamp: "Formatted at 2026-08-06 06:33:26 on dist-test-slave-6zbq"
server_key: "dc1b8d6f674cb07a5d435ed4a0bbb462"
server_key_iv: "e3343dc1f6ffd05ba583284de3846362"
server_key_version: "kuduclusterkey@0"
I20260806 06:33:26.040176 25900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:33:26.062762 25900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:33:26.064517 25900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:33:26.064641 25900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:33:26.064847 25900 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:33:26.065140 25900 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:33:26.065174 25900 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:26.065212 25900 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:33:26.065236 25900 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:26.085672 25900 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.67:34539
I20260806 06:33:26.086215 25900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
I20260806 06:33:26.087119 26079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.67:34539 every 8 connection(s)
I20260806 06:33:26.092850 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 25900
I20260806 06:33:26.092948 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-2/wal/instance
I20260806 06:33:26.093119 7341 external_mini_cluster.cc:1428] Setting key f631a7454d669a50776974fe8a919e48
Aug 06 06:33:26 dist-test-slave-6zbq krb5kdc[24709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998005, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.67@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:33:26.096170 25689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:26.087755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:41095 (local address 127.7.43.126:43181)
0806 06:33:26.087805 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:26.087807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:26.088160 (+ 353us) server_negotiation.cc:407] Connection header received
0806 06:33:26.088954 (+ 794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:26.088955 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:26.088968 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:26.088997 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:33:26.089741 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:26.090243 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:26.091206 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:26.091396 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:26.091453 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:26.093593 (+ 2140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:33:26.093594 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:33:26.093595 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:33:26.093601 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:33:26.094955 (+ 1354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:26.095305 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:26.095306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:26.095307 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:26.095356 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:26.095566 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:26.095567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:26.095568 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:26.095675 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:33:26.095731 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:26.095989 (+ 258us) server_negotiation.cc:299] Negotiation successful
0806 06:33:26.096071 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 06:33:26.096427 26084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:26.087356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:43181 (local address 127.7.43.67:41095)
0806 06:33:26.088007 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:26.088060 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 06:33:26.088773 (+ 713us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:33:26.089164 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:33:26.089167 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:33:26.089345 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:33:26.089603 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:26.089611 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:26.090331 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:26.090332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:26.091122 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:26.091125 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:26.091483 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:26.091484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:26.091600 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:26.092057 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:33:26.092065 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:33:26.093497 (+ 1432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:33:26.095032 (+ 1535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:33:26.095038 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:33:26.095040 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:33:26.095240 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:33:26.095411 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:33:26.095411 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:33:26.095412 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:33:26.095512 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:33:26.095729 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:33:26.095735 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:33:26.095874 (+ 139us) client_negotiation.cc:769] Sending connection context
0806 06:33:26.095960 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 06:33:26.096090 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":177,"threads_started":1}
I20260806 06:33:26.096834 26080 heartbeater.cc:347] Connected to a master server at 127.7.43.126:43181
I20260806 06:33:26.096923 26080 heartbeater.cc:464] Registering TS with master...
I20260806 06:33:26.097090 26080 heartbeater.cc:511] Master 127.7.43.126:43181 requested a full tablet report, sending...
I20260806 06:33:26.097427 25387 ts_manager.cc:194] Registered new tserver with Master: e10d1db4de26494c81b6bed2e25a2a50 (127.7.43.67:34539)
I20260806 06:33:26.097774 25387 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.67@KRBTEST.COM'} at 127.7.43.67:41095
I20260806 06:33:26.104573 7341 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:33:26.105312 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 25500
I20260806 06:33:26.139122 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 25692
I20260806 06:33:26.150302 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 25900
I20260806 06:33:26.155700 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 25339
I20260806 06:33:26.169075 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:43181
--webserver_interface=127.7.43.126
--webserver_port=32771
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:44991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.43.126:43181
--ranger_config_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.43.84:48483/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.43.84:48483/kms with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:33:26.248028 26086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:33:26.248224 26086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:33:26.248245 26086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:33:26.249326 26086 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:33:26.249370 26086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:33:26.249383 26086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:33:26.249394 26086 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:33:26.249406 26086 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:33:26.250944 26086 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:44991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.43.84:48483/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.43.126:43181
--ranger_config_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:43181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.43.126
--webserver_port=32771
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26086
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:33:26.251151 26086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:33:26.251345 26086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:33:26.255753 26098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:33:26.255735 26101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:33:26.255786 26096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:33:26.255916 26086 server_base.cc:1034] running on GCE node
I20260806 06:33:26.256115 26086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:33:26.256376 26086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:33:26.257519 26086 hybrid_clock.cc:648] HybridClock initialized: now 1785998006257507 us; error 32 us; skew 500 ppm
Aug 06 06:33:26 dist-test-slave-6zbq krb5kdc[24709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998006, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:33:26.259735 26086 init.cc:377] Logged in from keytab as kudu/127.7.43.126@KRBTEST.COM (short username kudu)
I20260806 06:33:26.260128 26086 webserver.cc:460] Webserver started at http://127.7.43.126:32771/ using document root <none> and password file <none>
I20260806 06:33:26.260327 26086 fs_manager.cc:359] Metadata directory not provided
I20260806 06:33:26.260381 26086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 06:33:26 dist-test-slave-6zbq krb5kdc[24709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998006, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.126@KRBTEST.COM for HTTP/127.7.43.84@KRBTEST.COM
I20260806 06:33:26.284962 26086 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:33:26.285609 26115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:26.285779 26086 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:26.285868 26086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "901f3fa5cdb240308ecdd5c870a2b433"
format_stamp: "Formatted at 2026-08-06 06:33:24 on dist-test-slave-6zbq"
server_key: "711b730f8eef114765a53e7f2802d255"
server_key_iv: "5c42c1310e6fbbfa6c15809f67139e58"
server_key_version: "kuduclusterkey@0"
I20260806 06:33:26.286221 26086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:33:26.305202 26086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:33:26.307112 26086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:33:26.307227 26086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:33:26.310989 26086 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.126:43181
I20260806 06:33:26.311050 26195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.126:43181 every 8 connection(s)
I20260806 06:33:26.311371 26086 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:33:26.312546 26197 sys_catalog.cc:263] Verifying existing consensus state
I20260806 06:33:26.313150 26197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 901f3fa5cdb240308ecdd5c870a2b433: Bootstrap starting.
I20260806 06:33:26.315228 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 26086
I20260806 06:33:26.315544 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.43.84:48483/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.65:36967
--local_ip_for_outbound_sockets=127.7.43.65
--tserver_master_addrs=127.7.43.126:43181
--webserver_port=39097
--webserver_interface=127.7.43.65
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab
--principal=kudu/127.7.43.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:44991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.43.84:48483/kms with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:33:26.321240 26197 log.cc:824] T 00000000000000000000000000000000 P 901f3fa5cdb240308ecdd5c870a2b433: Log is configured to *not* fsync() on all Append() calls
I20260806 06:33:26.322813 26197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 901f3fa5cdb240308ecdd5c870a2b433: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:33:26.323051 26197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 901f3fa5cdb240308ecdd5c870a2b433: Bootstrap complete.
I20260806 06:33:26.325173 26197 raft_consensus.cc:348] T 00000000000000000000000000000000 P 901f3fa5cdb240308ecdd5c870a2b433 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "901f3fa5cdb240308ecdd5c870a2b433" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 43181 } }
I20260806 06:33:26.325404 26197 raft_consensus.cc:729] T 00000000000000000000000000000000 P 901f3fa5cdb240308ecdd5c870a2b433 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 901f3fa5cdb240308ecdd5c870a2b433, State: Initialized, Role: FOLLOWER
I20260806 06:33:26.325536 26197 consensus_queue.cc:260] T 00000000000000000000000000000000 P 901f3fa5cdb240308ecdd5c870a2b433 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "901f3fa5cdb240308ecdd5c870a2b433" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 43181 } }
I20260806 06:33:26.325598 26197 raft_consensus.cc:388] T 00000000000000000000000000000000 P 901f3fa5cdb240308ecdd5c870a2b433 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:33:26.325636 26197 raft_consensus.cc:482] T 00000000000000000000000000000000 P 901f3fa5cdb240308ecdd5c870a2b433 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:33:26.325680 26197 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 901f3fa5cdb240308ecdd5c870a2b433 [term 1 FOLLOWER]: Advancing to term 2
I20260806 06:33:26.326653 26197 raft_consensus.cc:504] T 00000000000000000000000000000000 P 901f3fa5cdb240308ecdd5c870a2b433 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "901f3fa5cdb240308ecdd5c870a2b433" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 43181 } }
I20260806 06:33:26.326746 26197 leader_election.cc:302] T 00000000000000000000000000000000 P 901f3fa5cdb240308ecdd5c870a2b433 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 901f3fa5cdb240308ecdd5c870a2b433; no voters:
I20260806 06:33:26.326900 26197 leader_election.cc:288] T 00000000000000000000000000000000 P 901f3fa5cdb240308ecdd5c870a2b433 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 06:33:26.326946 26201 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 901f3fa5cdb240308ecdd5c870a2b433 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 06:33:26.327097 26201 raft_consensus.cc:686] T 00000000000000000000000000000000 P 901f3fa5cdb240308ecdd5c870a2b433 [term 2 LEADER]: Becoming Leader. State: Replica: 901f3fa5cdb240308ecdd5c870a2b433, State: Running, Role: LEADER
I20260806 06:33:26.327150 26197 sys_catalog.cc:564] T 00000000000000000000000000000000 P 901f3fa5cdb240308ecdd5c870a2b433 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:33:26.327237 26201 consensus_queue.cc:237] T 00000000000000000000000000000000 P 901f3fa5cdb240308ecdd5c870a2b433 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "901f3fa5cdb240308ecdd5c870a2b433" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 43181 } }
I20260806 06:33:26.327553 26201 sys_catalog.cc:455] T 00000000000000000000000000000000 P 901f3fa5cdb240308ecdd5c870a2b433 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 901f3fa5cdb240308ecdd5c870a2b433. Latest consensus state: current_term: 2 leader_uuid: "901f3fa5cdb240308ecdd5c870a2b433" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "901f3fa5cdb240308ecdd5c870a2b433" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 43181 } } }
I20260806 06:33:26.327641 26201 sys_catalog.cc:458] T 00000000000000000000000000000000 P 901f3fa5cdb240308ecdd5c870a2b433 [sys.catalog]: This master's current role is: LEADER
I20260806 06:33:26.327720 26203 sys_catalog.cc:455] T 00000000000000000000000000000000 P 901f3fa5cdb240308ecdd5c870a2b433 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "901f3fa5cdb240308ecdd5c870a2b433" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "901f3fa5cdb240308ecdd5c870a2b433" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 43181 } } }
I20260806 06:33:26.327795 26203 sys_catalog.cc:458] T 00000000000000000000000000000000 P 901f3fa5cdb240308ecdd5c870a2b433 [sys.catalog]: This master's current role is: LEADER
I20260806 06:33:26.327877 26209 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:33:26.328394 26209 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:33:26.328540 26209 catalog_manager.cc:1259] Loaded cluster ID: efe51c35a301468da7d0a86b50f19c88
I20260806 06:33:26.328581 26209 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:33:26.328835 26197 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 06:33:26.332320 26209 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:33:26.332845 26209 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 901f3fa5cdb240308ecdd5c870a2b433: Loaded TSK: 0
I20260806 06:33:26.333168 26209 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 06:33:26.429131 26199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:33:26.429469 26199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:33:26.429549 26199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:33:26.431283 26199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:33:26.431378 26199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:33:26.431455 26199 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.65
I20260806 06:33:26.433558 26199 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:44991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.43.84:48483/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab
--principal=kudu/127.7.43.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.65:36967
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.43.65
--webserver_port=39097
--tserver_master_addrs=127.7.43.126:43181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26199
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.65
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:33:26.433962 26199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:33:26.434365 26199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:33:26.440769 26256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:33:26.440745 26252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:33:26.441028 26254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:33:26.441066 26199 server_base.cc:1034] running on GCE node
I20260806 06:33:26.441340 26199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:33:26.441565 26199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:33:26.442715 26199 hybrid_clock.cc:648] HybridClock initialized: now 1785998006442697 us; error 35 us; skew 500 ppm
Aug 06 06:33:26 dist-test-slave-6zbq krb5kdc[24709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998006, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:33:26.445055 26199 init.cc:377] Logged in from keytab as kudu/127.7.43.65@KRBTEST.COM (short username kudu)
I20260806 06:33:26.445473 26199 webserver.cc:460] Webserver started at http://127.7.43.65:39097/ using document root <none> and password file <none>
I20260806 06:33:26.445683 26199 fs_manager.cc:359] Metadata directory not provided
I20260806 06:33:26.445746 26199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 06:33:26 dist-test-slave-6zbq krb5kdc[24709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998006, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.65@KRBTEST.COM for HTTP/127.7.43.84@KRBTEST.COM
I20260806 06:33:26.470762 26199 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.001s sys 0.000s
I20260806 06:33:26.474993 26265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:26.475216 26199 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.000s
I20260806 06:33:26.475306 26199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-0/wal
uuid: "62ff3045c51d4c7aacdff1226a654dc2"
format_stamp: "Formatted at 2026-08-06 06:33:25 on dist-test-slave-6zbq"
server_key: "ae3c9e8e2c8fe1285bb0d9e2cbb344e7"
server_key_iv: "47dc7f7c8b4b3df31f2e53e2c8da5d06"
server_key_version: "kuduclusterkey@0"
I20260806 06:33:26.475725 26199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:33:26.507035 26199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:33:26.508993 26199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:33:26.509171 26199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:33:26.509411 26199 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:33:26.509758 26199 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:33:26.509828 26199 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:26.509872 26199 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:33:26.509904 26199 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:26.516295 26199 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.65:36967
I20260806 06:33:26.516366 26435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.65:36967 every 8 connection(s)
I20260806 06:33:26.516810 26199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
Aug 06 06:33:26 dist-test-slave-6zbq krb5kdc[24709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998006, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.65@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:33:26.525353 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 26199
I20260806 06:33:26.525663 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.43.84:48483/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.66:36295
--local_ip_for_outbound_sockets=127.7.43.66
--tserver_master_addrs=127.7.43.126:43181
--webserver_port=34803
--webserver_interface=127.7.43.66
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab
--principal=kudu/127.7.43.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:44991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.43.84:48483/kms with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:33:26.527973 26442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:26.517495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:43181 (local address 127.7.43.65:52825)
0806 06:33:26.517837 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:26.517867 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 06:33:26.518407 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:33:26.518864 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:33:26.518868 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:33:26.519038 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:33:26.519221 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:26.519225 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:26.520283 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:26.520284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:26.521065 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:26.521068 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:26.521966 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:26.521967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:26.522108 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:26.522647 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:33:26.522657 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:33:26.524052 (+ 1395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:33:26.526090 (+ 2038us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:33:26.526097 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:33:26.526099 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:33:26.526411 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:33:26.526715 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:33:26.526716 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:33:26.526716 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:33:26.526844 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:33:26.527335 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:33:26.527341 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:33:26.527489 (+ 148us) client_negotiation.cc:769] Sending connection context
0806 06:33:26.527595 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 06:33:26.527757 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":82,"threads_started":1}
I20260806 06:33:26.528110 26443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:26.517584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:52825 (local address 127.7.43.126:43181)
0806 06:33:26.517771 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:26.517779 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:26.517971 (+ 192us) server_negotiation.cc:407] Connection header received
0806 06:33:26.518631 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:26.518636 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:26.518683 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:26.518812 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:33:26.519333 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:26.520186 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:26.521168 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:26.521827 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:26.521956 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:26.524208 (+ 2252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:33:26.524213 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:33:26.524213 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:33:26.524225 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:33:26.526013 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:26.526509 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:26.526512 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:26.526513 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:26.526624 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:26.526924 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:26.526925 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:26.526925 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:26.527218 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:33:26.527303 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:26.527734 (+ 431us) server_negotiation.cc:299] Negotiation successful
0806 06:33:26.527907 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":72,"threads_started":1}
I20260806 06:33:26.528997 26437 heartbeater.cc:347] Connected to a master server at 127.7.43.126:43181
I20260806 06:33:26.529093 26437 heartbeater.cc:464] Registering TS with master...
I20260806 06:33:26.529265 26437 heartbeater.cc:511] Master 127.7.43.126:43181 requested a full tablet report, sending...
W20260806 06:33:26.530102 26437 heartbeater.cc:644] Failed to heartbeat to 127.7.43.126:43181 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 06:33:26.530167 26437 heartbeater.cc:464] Registering TS with master...
I20260806 06:33:26.530227 26437 heartbeater.cc:511] Master 127.7.43.126:43181 requested a full tablet report, sending...
I20260806 06:33:26.530580 26437 heartbeater.cc:464] Registering TS with master...
I20260806 06:33:26.530653 26437 heartbeater.cc:511] Master 127.7.43.126:43181 requested a full tablet report, sending...
W20260806 06:33:26.530961 26437 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 06:33:26.641855 26444 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:33:26.642115 26444 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:33:26.642153 26444 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:33:26.643815 26444 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:33:26.643873 26444 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:33:26.643910 26444 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.66
I20260806 06:33:26.646045 26444 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:44991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.43.84:48483/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab
--principal=kudu/127.7.43.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.66:36295
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.43.66
--webserver_port=34803
--tserver_master_addrs=127.7.43.126:43181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26444
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.66
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:33:26.646281 26444 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:33:26.646576 26444 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:33:26.652072 26454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:33:26.652271 26444 server_base.cc:1034] running on GCE node
W20260806 06:33:26.652076 26458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:33:26.652105 26452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:33:26.652565 26444 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:33:26.652833 26444 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:33:26.653987 26444 hybrid_clock.cc:648] HybridClock initialized: now 1785998006653961 us; error 44 us; skew 500 ppm
Aug 06 06:33:26 dist-test-slave-6zbq krb5kdc[24709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998006, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:33:26.656386 26444 init.cc:377] Logged in from keytab as kudu/127.7.43.66@KRBTEST.COM (short username kudu)
I20260806 06:33:26.656764 26444 webserver.cc:460] Webserver started at http://127.7.43.66:34803/ using document root <none> and password file <none>
I20260806 06:33:26.656963 26444 fs_manager.cc:359] Metadata directory not provided
I20260806 06:33:26.657024 26444 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 06:33:26 dist-test-slave-6zbq krb5kdc[24709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998006, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.66@KRBTEST.COM for HTTP/127.7.43.84@KRBTEST.COM
I20260806 06:33:26.671591 26444 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:33:26.672557 26472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:26.672775 26444 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 06:33:26.672869 26444 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-1/wal
uuid: "49a8501de59748449a52600db82ef49f"
format_stamp: "Formatted at 2026-08-06 06:33:25 on dist-test-slave-6zbq"
server_key: "bc5b91b08c153981957f8b4d6d4efc84"
server_key_iv: "cae249eb985f8de762a4d83469903389"
server_key_version: "kuduclusterkey@0"
I20260806 06:33:26.673260 26444 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:33:26.709576 26444 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:33:26.711448 26444 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:33:26.711598 26444 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:33:26.711848 26444 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:33:26.712216 26444 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:33:26.712276 26444 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:26.712304 26444 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:33:26.712329 26444 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:26.719741 26444 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.66:36295
I20260806 06:33:26.719795 26623 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.66:36295 every 8 connection(s)
I20260806 06:33:26.720227 26444 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
I20260806 06:33:26.726130 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 26444
I20260806 06:33:26.726444 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.43.84:48483/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.67:34539
--local_ip_for_outbound_sockets=127.7.43.67
--tserver_master_addrs=127.7.43.126:43181
--webserver_port=42371
--webserver_interface=127.7.43.67
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab
--principal=kudu/127.7.43.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:44991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.43.84:48483/kms with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 06:33:26 dist-test-slave-6zbq krb5kdc[24709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998006, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.66@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:33:26.730398 26443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:26.720941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:57149 (local address 127.7.43.126:43181)
0806 06:33:26.720986 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:26.720988 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:26.721232 (+ 244us) server_negotiation.cc:407] Connection header received
0806 06:33:26.721923 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:26.721924 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:26.721935 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:26.721986 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:33:26.722659 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:26.723134 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:26.724560 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:26.724757 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:26.724806 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:26.727171 (+ 2365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:33:26.727173 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:33:26.727173 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:33:26.727181 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:33:26.728638 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:26.729193 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:26.729194 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:26.729195 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:26.729313 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:26.729682 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:26.729684 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:26.729684 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:26.729814 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:33:26.729882 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:26.730205 (+ 323us) server_negotiation.cc:299] Negotiation successful
0806 06:33:26.730286 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:33:26.730557 26628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:26.720927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:43181 (local address 127.7.43.66:57149)
0806 06:33:26.721083 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:26.721145 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 06:33:26.721804 (+ 659us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:33:26.722065 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:33:26.722069 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:33:26.722276 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:33:26.722571 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:26.722578 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:26.723234 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:26.723235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:26.724460 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:26.724463 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:26.724853 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:26.724855 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:26.724994 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:26.725423 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:33:26.725433 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:33:26.727072 (+ 1639us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:33:26.728727 (+ 1655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:33:26.728734 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:33:26.728737 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:33:26.729094 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:33:26.729393 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:33:26.729394 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:33:26.729394 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:33:26.729583 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:33:26.729896 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:33:26.729902 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:33:26.730054 (+ 152us) client_negotiation.cc:769] Sending connection context
0806 06:33:26.730168 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 06:33:26.730314 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":78,"threads_started":1}
I20260806 06:33:26.731200 26624 heartbeater.cc:347] Connected to a master server at 127.7.43.126:43181
I20260806 06:33:26.731303 26624 heartbeater.cc:464] Registering TS with master...
I20260806 06:33:26.731480 26624 heartbeater.cc:511] Master 127.7.43.126:43181 requested a full tablet report, sending...
W20260806 06:33:26.732203 26624 heartbeater.cc:644] Failed to heartbeat to 127.7.43.126:43181 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 06:33:26.732275 26624 heartbeater.cc:464] Registering TS with master...
I20260806 06:33:26.732337 26624 heartbeater.cc:511] Master 127.7.43.126:43181 requested a full tablet report, sending...
I20260806 06:33:26.732677 26624 heartbeater.cc:464] Registering TS with master...
I20260806 06:33:26.732756 26624 heartbeater.cc:511] Master 127.7.43.126:43181 requested a full tablet report, sending...
W20260806 06:33:26.733101 26624 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 06:33:26.825438 26629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:33:26.825624 26629 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:33:26.825654 26629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:33:26.826826 26629 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:33:26.826866 26629 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:33:26.826901 26629 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.67
I20260806 06:33:26.828634 26629 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:44991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.43.84:48483/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab
--principal=kudu/127.7.43.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.67:34539
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.43.67
--webserver_port=42371
--tserver_master_addrs=127.7.43.126:43181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26629
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.67
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:33:26.828859 26629 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:33:26.829068 26629 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:33:26.833693 26640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:33:26.833693 26645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:33:26.833789 26641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:33:26.834488 26629 server_base.cc:1034] running on GCE node
I20260806 06:33:26.834682 26629 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:33:26.834932 26629 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:33:26.836118 26629 hybrid_clock.cc:648] HybridClock initialized: now 1785998006836099 us; error 32 us; skew 500 ppm
Aug 06 06:33:26 dist-test-slave-6zbq krb5kdc[24709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998006, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:33:26.839490 26629 init.cc:377] Logged in from keytab as kudu/127.7.43.67@KRBTEST.COM (short username kudu)
I20260806 06:33:26.839864 26629 webserver.cc:460] Webserver started at http://127.7.43.67:42371/ using document root <none> and password file <none>
I20260806 06:33:26.840039 26629 fs_manager.cc:359] Metadata directory not provided
I20260806 06:33:26.840087 26629 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 06:33:26 dist-test-slave-6zbq krb5kdc[24709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998006, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.67@KRBTEST.COM for HTTP/127.7.43.84@KRBTEST.COM
I20260806 06:33:26.857601 26629 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:33:26.859402 26658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:26.859596 26629 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260806 06:33:26.859702 26629 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-2/wal
uuid: "e10d1db4de26494c81b6bed2e25a2a50"
format_stamp: "Formatted at 2026-08-06 06:33:26 on dist-test-slave-6zbq"
server_key: "dc1b8d6f674cb07a5d435ed4a0bbb462"
server_key_iv: "e3343dc1f6ffd05ba583284de3846362"
server_key_version: "kuduclusterkey@0"
I20260806 06:33:26.860041 26629 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:33:26.872067 26629 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:33:26.873965 26629 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:33:26.874099 26629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:33:26.874294 26629 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:33:26.874631 26629 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:33:26.874675 26629 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:26.874702 26629 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:33:26.874743 26629 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:26.881870 26629 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.67:34539
I20260806 06:33:26.881904 26815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.67:34539 every 8 connection(s)
I20260806 06:33:26.882390 26629 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
I20260806 06:33:26.887115 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 26629
Aug 06 06:33:26 dist-test-slave-6zbq krb5kdc[24709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998006, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.67@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:33:26.892849 26443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:26.883192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:36697 (local address 127.7.43.126:43181)
0806 06:33:26.883237 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:26.883239 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:26.883535 (+ 296us) server_negotiation.cc:407] Connection header received
0806 06:33:26.884187 (+ 652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:26.884189 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:26.884201 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:26.884231 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:33:26.885038 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:26.885789 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:26.887227 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:26.887483 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:26.887552 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:26.889808 (+ 2256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:33:26.889810 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:33:26.889810 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:33:26.889817 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:33:26.891336 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:26.891801 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:26.891802 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:26.891803 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:26.891864 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:26.892141 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:26.892142 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:26.892142 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:26.892286 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:33:26.892360 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:26.892675 (+ 315us) server_negotiation.cc:299] Negotiation successful
0806 06:33:26.892760 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:33:26.893088 26819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:26.883209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:43181 (local address 127.7.43.67:36697)
0806 06:33:26.883403 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:26.883447 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 06:33:26.884061 (+ 614us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:33:26.884434 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:33:26.884438 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:33:26.884649 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:33:26.884923 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:26.884930 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:26.885911 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:26.885912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:26.887109 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:26.887112 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:26.887596 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:26.887598 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:26.887753 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:26.888207 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:33:26.888217 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:33:26.889721 (+ 1504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:33:26.891425 (+ 1704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:33:26.891431 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:33:26.891433 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:33:26.891711 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:33:26.891935 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:33:26.891936 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:33:26.891936 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:33:26.892060 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:33:26.892358 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:33:26.892363 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:33:26.892510 (+ 147us) client_negotiation.cc:769] Sending connection context
0806 06:33:26.892622 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 06:33:26.892763 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":94,"threads_started":1}
I20260806 06:33:26.893524 26816 heartbeater.cc:347] Connected to a master server at 127.7.43.126:43181
I20260806 06:33:26.893625 26816 heartbeater.cc:464] Registering TS with master...
I20260806 06:33:26.893805 26816 heartbeater.cc:511] Master 127.7.43.126:43181 requested a full tablet report, sending...
W20260806 06:33:26.894558 26816 heartbeater.cc:644] Failed to heartbeat to 127.7.43.126:43181 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 06:33:26.894625 26816 heartbeater.cc:464] Registering TS with master...
I20260806 06:33:26.894694 26816 heartbeater.cc:511] Master 127.7.43.126:43181 requested a full tablet report, sending...
I20260806 06:33:26.895383 26816 heartbeater.cc:464] Registering TS with master...
I20260806 06:33:26.895457 26816 heartbeater.cc:511] Master 127.7.43.126:43181 requested a full tablet report, sending...
W20260806 06:33:26.895807 26816 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 06:33:26.901367 26822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:26.887685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48866 (local address 127.7.43.126:43181)
0806 06:33:26.887841 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:26.887843 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:26.887848 (+ 5us) server_negotiation.cc:407] Connection header received
0806 06:33:26.887863 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:26.887863 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:26.887870 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:26.887924 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:33:26.888517 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:26.889074 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:26.890048 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:26.890318 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:26.890403 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:26.896563 (+ 6160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:33:26.896564 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:33:26.896565 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:33:26.896572 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:33:26.899740 (+ 3168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:26.900156 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:26.900157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:26.900157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:26.900220 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:26.900438 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:26.900439 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:26.900439 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:26.900557 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:33:26.901140 (+ 583us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:26.901186 (+ 46us) server_negotiation.cc:299] Negotiation successful
0806 06:33:26.901272 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":52,"threads_started":1}
Aug 06 06:33:27 dist-test-slave-6zbq krb5kdc[24709](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785998007, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:33:27.531132 26437 heartbeater.cc:464] Registering TS with master...
I20260806 06:33:27.531301 26437 heartbeater.cc:511] Master 127.7.43.126:43181 requested a full tablet report, sending...
I20260806 06:33:27.531749 26139 ts_manager.cc:194] Registered new tserver with Master: 62ff3045c51d4c7aacdff1226a654dc2 (127.7.43.65:36967)
I20260806 06:33:27.532780 26139 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.65@KRBTEST.COM'} at 127.7.43.65:52825
I20260806 06:33:27.733253 26624 heartbeater.cc:464] Registering TS with master...
I20260806 06:33:27.733395 26624 heartbeater.cc:511] Master 127.7.43.126:43181 requested a full tablet report, sending...
I20260806 06:33:27.733701 26139 ts_manager.cc:194] Registered new tserver with Master: 49a8501de59748449a52600db82ef49f (127.7.43.66:36295)
I20260806 06:33:27.734256 26139 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.66@KRBTEST.COM'} at 127.7.43.66:57149
Aug 06 06:33:27 dist-test-slave-6zbq krb5kdc[24709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998007, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.126@KRBTEST.COM for HTTP/127.7.43.84@KRBTEST.COM
I20260806 06:33:27.895972 26816 heartbeater.cc:464] Registering TS with master...
I20260806 06:33:27.896113 26816 heartbeater.cc:511] Master 127.7.43.126:43181 requested a full tablet report, sending...
I20260806 06:33:27.896389 26139 ts_manager.cc:194] Registered new tserver with Master: e10d1db4de26494c81b6bed2e25a2a50 (127.7.43.67:34539)
I20260806 06:33:27.896770 26139 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.67@KRBTEST.COM'} at 127.7.43.67:36697
I20260806 06:33:27.904457 7341 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:33:27.911149 26853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:27.905337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48878 (local address 127.7.43.126:43181)
0806 06:33:27.905436 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:27.905438 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:27.905476 (+ 38us) server_negotiation.cc:407] Connection header received
0806 06:33:27.905547 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:27.905548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:27.905559 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:27.905616 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:33:27.906157 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:27.906709 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:27.907404 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:27.907603 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:27.907657 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:27.908796 (+ 1139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:33:27.908797 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:33:27.908798 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:33:27.908805 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:33:27.910117 (+ 1312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:27.910458 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:27.910459 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:27.910459 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:27.910515 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:27.910687 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:27.910688 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:27.910688 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:27.910790 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:33:27.910877 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:27.910974 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 06:33:27.911053 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":54,"threads_started":1}
I20260806 06:33:27.912889 26139 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48878:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:33:27.914530 26139 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:33:27.921702 26862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:27.917095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:36967 (local address 127.0.0.1:53728)
0806 06:33:27.918705 (+ 1610us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:27.918711 (+ 6us) client_negotiation.cc:174] Beginning negotiation
0806 06:33:27.918751 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:33:27.918908 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:33:27.918910 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:33:27.918912 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:33:27.918983 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:27.918985 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:27.919784 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:27.919785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:27.920707 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:27.920710 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:27.921414 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:27.921415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:27.921493 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:27.921506 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 06:33:27.921569 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 06:33:27.921618 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1596,"mutex_wait_us":2,"thread_start_us":82,"threads_started":1}
I20260806 06:33:27.921743 26868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:27.917527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53728 (local address 127.7.43.65:36967)
0806 06:33:27.917633 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:27.917637 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:27.918766 (+ 1129us) server_negotiation.cc:407] Connection header received
0806 06:33:27.918807 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:27.918810 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:27.918825 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:27.918889 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:33:27.919044 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:27.919710 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:27.920780 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:27.921336 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:27.921401 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:27.921462 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:27.921589 (+ 127us) server_negotiation.cc:299] Negotiation successful
0806 06:33:27.921630 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":60,"threads_started":1}
I20260806 06:33:27.922567 26340 tablet_service.cc:1467] Processing CreateTablet for tablet ee1526a6d25f408ebc63bb6e6a6f0cbf (DEFAULT_TABLE table=test-table [id=89060829aae345f4aa24e2d22f03bea6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:33:27.922951 26340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee1526a6d25f408ebc63bb6e6a6f0cbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:33:27.926563 26862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:27.917321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.66:36295 (local address 127.0.0.1:60902)
0806 06:33:27.921976 (+ 4655us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:27.921985 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:33:27.922016 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:33:27.922690 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:33:27.922692 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:33:27.922694 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:33:27.922726 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:27.922727 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:27.923936 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:27.923937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:27.924920 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:27.924924 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:27.926317 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:27.926318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:27.926397 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:27.926400 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:33:27.926453 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 06:33:27.926498 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4645,"thread_start_us":75,"threads_started":1}
I20260806 06:33:27.926579 26867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:27.917436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60902 (local address 127.7.43.66:36295)
0806 06:33:27.922427 (+ 4991us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:27.922432 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:27.922443 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:33:27.922478 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:27.922480 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:27.922497 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:27.922593 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:33:27.922809 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:27.923841 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:27.925020 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:27.926240 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:27.926306 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:27.926391 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:27.926466 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 06:33:27.926502 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4955,"thread_start_us":51,"threads_started":1}
I20260806 06:33:27.927490 26534 tablet_service.cc:1467] Processing CreateTablet for tablet ee1526a6d25f408ebc63bb6e6a6f0cbf (DEFAULT_TABLE table=test-table [id=89060829aae345f4aa24e2d22f03bea6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:33:27.927784 26534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee1526a6d25f408ebc63bb6e6a6f0cbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:33:27.930495 26863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:27.917094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.67:34539 (local address 127.0.0.1:43286)
0806 06:33:27.918497 (+ 1403us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:27.918539 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 06:33:27.918596 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:33:27.927014 (+ 8418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:33:27.927016 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:33:27.927018 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:33:27.927055 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:27.927057 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:27.928237 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:27.928238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:27.929305 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:27.929308 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:27.930264 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:27.930265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:27.930323 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:27.930327 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:33:27.930395 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 06:33:27.930432 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1371,"thread_start_us":91,"threads_started":1}
I20260806 06:33:27.930864 26865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:27.917345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43286 (local address 127.7.43.67:34539)
0806 06:33:27.926822 (+ 9477us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:27.926827 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:27.926837 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:33:27.926870 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:27.926872 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:27.926888 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:27.927016 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:33:27.927114 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:27.928132 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:27.929399 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:27.930111 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:27.930175 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:27.930226 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:27.930750 (+ 524us) server_negotiation.cc:299] Negotiation successful
0806 06:33:27.930792 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9440,"thread_start_us":51,"threads_started":1}
I20260806 06:33:27.931566 26724 tablet_service.cc:1467] Processing CreateTablet for tablet ee1526a6d25f408ebc63bb6e6a6f0cbf (DEFAULT_TABLE table=test-table [id=89060829aae345f4aa24e2d22f03bea6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:33:27.931879 26724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee1526a6d25f408ebc63bb6e6a6f0cbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:33:27.934244 26876 tablet_bootstrap.cc:492] T ee1526a6d25f408ebc63bb6e6a6f0cbf P e10d1db4de26494c81b6bed2e25a2a50: Bootstrap starting.
I20260806 06:33:27.934485 26870 tablet_bootstrap.cc:492] T ee1526a6d25f408ebc63bb6e6a6f0cbf P 62ff3045c51d4c7aacdff1226a654dc2: Bootstrap starting.
I20260806 06:33:27.935170 26870 tablet_bootstrap.cc:654] T ee1526a6d25f408ebc63bb6e6a6f0cbf P 62ff3045c51d4c7aacdff1226a654dc2: Neither blocks nor log segments found. Creating new log.
I20260806 06:33:27.935170 26876 tablet_bootstrap.cc:654] T ee1526a6d25f408ebc63bb6e6a6f0cbf P e10d1db4de26494c81b6bed2e25a2a50: Neither blocks nor log segments found. Creating new log.
I20260806 06:33:27.935491 26876 log.cc:824] T ee1526a6d25f408ebc63bb6e6a6f0cbf P e10d1db4de26494c81b6bed2e25a2a50: Log is configured to *not* fsync() on all Append() calls
I20260806 06:33:27.935537 26870 log.cc:824] T ee1526a6d25f408ebc63bb6e6a6f0cbf P 62ff3045c51d4c7aacdff1226a654dc2: Log is configured to *not* fsync() on all Append() calls
I20260806 06:33:27.935847 26872 tablet_bootstrap.cc:492] T ee1526a6d25f408ebc63bb6e6a6f0cbf P 49a8501de59748449a52600db82ef49f: Bootstrap starting.
I20260806 06:33:27.936326 26872 tablet_bootstrap.cc:654] T ee1526a6d25f408ebc63bb6e6a6f0cbf P 49a8501de59748449a52600db82ef49f: Neither blocks nor log segments found. Creating new log.
I20260806 06:33:27.936616 26872 log.cc:824] T ee1526a6d25f408ebc63bb6e6a6f0cbf P 49a8501de59748449a52600db82ef49f: Log is configured to *not* fsync() on all Append() calls
I20260806 06:33:27.937233 26872 tablet_bootstrap.cc:492] T ee1526a6d25f408ebc63bb6e6a6f0cbf P 49a8501de59748449a52600db82ef49f: No bootstrap required, opened a new log
I20260806 06:33:27.937292 26872 ts_tablet_manager.cc:1397] T ee1526a6d25f408ebc63bb6e6a6f0cbf P 49a8501de59748449a52600db82ef49f: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:33:27.938939 26870 tablet_bootstrap.cc:492] T ee1526a6d25f408ebc63bb6e6a6f0cbf P 62ff3045c51d4c7aacdff1226a654dc2: No bootstrap required, opened a new log
I20260806 06:33:27.939003 26870 ts_tablet_manager.cc:1397] T ee1526a6d25f408ebc63bb6e6a6f0cbf P 62ff3045c51d4c7aacdff1226a654dc2: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260806 06:33:27.939100 26876 tablet_bootstrap.cc:492] T ee1526a6d25f408ebc63bb6e6a6f0cbf P e10d1db4de26494c81b6bed2e25a2a50: No bootstrap required, opened a new log
I20260806 06:33:27.939146 26876 ts_tablet_manager.cc:1397] T ee1526a6d25f408ebc63bb6e6a6f0cbf P e10d1db4de26494c81b6bed2e25a2a50: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260806 06:33:27.941525 26876 raft_consensus.cc:348] T ee1526a6d25f408ebc63bb6e6a6f0cbf P e10d1db4de26494c81b6bed2e25a2a50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62ff3045c51d4c7aacdff1226a654dc2" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36967 } } peers { permanent_uuid: "e10d1db4de26494c81b6bed2e25a2a50" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 34539 } } peers { permanent_uuid: "49a8501de59748449a52600db82ef49f" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 36295 } }
I20260806 06:33:27.941525 26870 raft_consensus.cc:348] T ee1526a6d25f408ebc63bb6e6a6f0cbf P 62ff3045c51d4c7aacdff1226a654dc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62ff3045c51d4c7aacdff1226a654dc2" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36967 } } peers { permanent_uuid: "e10d1db4de26494c81b6bed2e25a2a50" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 34539 } } peers { permanent_uuid: "49a8501de59748449a52600db82ef49f" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 36295 } }
I20260806 06:33:27.941677 26876 raft_consensus.cc:374] T ee1526a6d25f408ebc63bb6e6a6f0cbf P e10d1db4de26494c81b6bed2e25a2a50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:33:27.941677 26870 raft_consensus.cc:374] T ee1526a6d25f408ebc63bb6e6a6f0cbf P 62ff3045c51d4c7aacdff1226a654dc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:33:27.941712 26876 raft_consensus.cc:729] T ee1526a6d25f408ebc63bb6e6a6f0cbf P e10d1db4de26494c81b6bed2e25a2a50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e10d1db4de26494c81b6bed2e25a2a50, State: Initialized, Role: FOLLOWER
I20260806 06:33:27.941720 26870 raft_consensus.cc:729] T ee1526a6d25f408ebc63bb6e6a6f0cbf P 62ff3045c51d4c7aacdff1226a654dc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62ff3045c51d4c7aacdff1226a654dc2, State: Initialized, Role: FOLLOWER
I20260806 06:33:27.941820 26876 consensus_queue.cc:260] T ee1526a6d25f408ebc63bb6e6a6f0cbf P e10d1db4de26494c81b6bed2e25a2a50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62ff3045c51d4c7aacdff1226a654dc2" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36967 } } peers { permanent_uuid: "e10d1db4de26494c81b6bed2e25a2a50" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 34539 } } peers { permanent_uuid: "49a8501de59748449a52600db82ef49f" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 36295 } }
I20260806 06:33:27.941820 26870 consensus_queue.cc:260] T ee1526a6d25f408ebc63bb6e6a6f0cbf P 62ff3045c51d4c7aacdff1226a654dc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62ff3045c51d4c7aacdff1226a654dc2" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36967 } } peers { permanent_uuid: "e10d1db4de26494c81b6bed2e25a2a50" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 34539 } } peers { permanent_uuid: "49a8501de59748449a52600db82ef49f" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 36295 } }
I20260806 06:33:27.942072 26870 ts_tablet_manager.cc:1428] T ee1526a6d25f408ebc63bb6e6a6f0cbf P 62ff3045c51d4c7aacdff1226a654dc2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:33:27.942072 26876 ts_tablet_manager.cc:1428] T ee1526a6d25f408ebc63bb6e6a6f0cbf P e10d1db4de26494c81b6bed2e25a2a50: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:33:27.942276 26437 heartbeater.cc:503] Master 127.7.43.126:43181 was elected leader, sending a full tablet report...
I20260806 06:33:27.942787 26816 heartbeater.cc:503] Master 127.7.43.126:43181 was elected leader, sending a full tablet report...
I20260806 06:33:27.943527 26872 raft_consensus.cc:348] T ee1526a6d25f408ebc63bb6e6a6f0cbf P 49a8501de59748449a52600db82ef49f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62ff3045c51d4c7aacdff1226a654dc2" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36967 } } peers { permanent_uuid: "e10d1db4de26494c81b6bed2e25a2a50" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 34539 } } peers { permanent_uuid: "49a8501de59748449a52600db82ef49f" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 36295 } }
I20260806 06:33:27.943645 26872 raft_consensus.cc:374] T ee1526a6d25f408ebc63bb6e6a6f0cbf P 49a8501de59748449a52600db82ef49f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:33:27.943675 26872 raft_consensus.cc:729] T ee1526a6d25f408ebc63bb6e6a6f0cbf P 49a8501de59748449a52600db82ef49f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49a8501de59748449a52600db82ef49f, State: Initialized, Role: FOLLOWER
I20260806 06:33:27.943825 26872 consensus_queue.cc:260] T ee1526a6d25f408ebc63bb6e6a6f0cbf P 49a8501de59748449a52600db82ef49f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62ff3045c51d4c7aacdff1226a654dc2" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36967 } } peers { permanent_uuid: "e10d1db4de26494c81b6bed2e25a2a50" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 34539 } } peers { permanent_uuid: "49a8501de59748449a52600db82ef49f" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 36295 } }
I20260806 06:33:27.944072 26872 ts_tablet_manager.cc:1428] T ee1526a6d25f408ebc63bb6e6a6f0cbf P 49a8501de59748449a52600db82ef49f: Time spent starting tablet: real 0.007s user 0.000s sys 0.003s
I20260806 06:33:27.944124 26624 heartbeater.cc:503] Master 127.7.43.126:43181 was elected leader, sending a full tablet report...
W20260806 06:33:27.971969 26625 tablet.cc:2369] T ee1526a6d25f408ebc63bb6e6a6f0cbf P 49a8501de59748449a52600db82ef49f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 06:33:28.018275 26439 tablet.cc:2369] T ee1526a6d25f408ebc63bb6e6a6f0cbf P 62ff3045c51d4c7aacdff1226a654dc2: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 06:33:28 dist-test-slave-6zbq krb5kdc[24709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998007, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.126@KRBTEST.COM for HTTP/127.7.43.84@KRBTEST.COM
W20260806 06:33:28.133867 26817 tablet.cc:2369] T ee1526a6d25f408ebc63bb6e6a6f0cbf P e10d1db4de26494c81b6bed2e25a2a50: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:33:28.267452 26884 raft_consensus.cc:482] T ee1526a6d25f408ebc63bb6e6a6f0cbf P e10d1db4de26494c81b6bed2e25a2a50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:33:28.267611 26884 raft_consensus.cc:504] T ee1526a6d25f408ebc63bb6e6a6f0cbf P e10d1db4de26494c81b6bed2e25a2a50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62ff3045c51d4c7aacdff1226a654dc2" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36967 } } peers { permanent_uuid: "e10d1db4de26494c81b6bed2e25a2a50" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 34539 } } peers { permanent_uuid: "49a8501de59748449a52600db82ef49f" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 36295 } }
I20260806 06:33:28.268082 26884 leader_election.cc:288] T ee1526a6d25f408ebc63bb6e6a6f0cbf P e10d1db4de26494c81b6bed2e25a2a50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 62ff3045c51d4c7aacdff1226a654dc2 (127.7.43.65:36967), 49a8501de59748449a52600db82ef49f (127.7.43.66:36295)
I20260806 06:33:28.271695 26897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:28.268275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:36967 (local address 127.7.43.67:43709)
0806 06:33:28.268488 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:28.268502 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:33:28.268581 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:33:28.268821 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:33:28.268822 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:33:28.268823 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:33:28.268902 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:28.268904 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:28.269724 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:28.269726 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:28.270623 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:28.270626 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:28.271395 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:28.271396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:28.271480 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:28.271485 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:33:28.271540 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 06:33:28.271584 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":79,"threads_started":1}
I20260806 06:33:28.272056 26868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:28.268376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:43709 (local address 127.7.43.65:36967)
0806 06:33:28.268602 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:28.268604 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:28.268617 (+ 13us) server_negotiation.cc:407] Connection header received
0806 06:33:28.268648 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:28.268649 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:28.268663 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:28.268726 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:33:28.269045 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:28.269632 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:28.270713 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:28.271268 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:28.271331 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:28.271343 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:28.271975 (+ 632us) server_negotiation.cc:299] Negotiation successful
0806 06:33:28.272009 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175}
I20260806 06:33:28.272375 26867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:28.268840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:36455 (local address 127.7.43.66:36295)
0806 06:33:28.268899 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:28.268901 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:28.268908 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:33:28.268933 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:28.268933 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:28.268943 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:28.268994 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:33:28.269316 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:28.270056 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:28.271456 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:28.271995 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:28.272042 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:28.272051 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:28.272269 (+ 218us) server_negotiation.cc:299] Negotiation successful
0806 06:33:28.272309 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 06:33:28.272433 26372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee1526a6d25f408ebc63bb6e6a6f0cbf" candidate_uuid: "e10d1db4de26494c81b6bed2e25a2a50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62ff3045c51d4c7aacdff1226a654dc2" is_pre_election: true
I20260806 06:33:28.272579 26901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:28.268456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.66:36295 (local address 127.7.43.67:36455)
0806 06:33:28.268683 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:28.268691 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:33:28.268725 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:33:28.269157 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:33:28.269159 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:33:28.269161 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:33:28.269221 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:28.269223 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:28.270170 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:28.270171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:28.271348 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:28.271352 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:28.272107 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:28.272108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:28.272193 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:28.272196 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:33:28.272464 (+ 268us) client_negotiation.cc:240] Negotiation successful
0806 06:33:28.272505 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":65,"threads_started":1}
I20260806 06:33:28.272646 26372 raft_consensus.cc:2454] T ee1526a6d25f408ebc63bb6e6a6f0cbf P 62ff3045c51d4c7aacdff1226a654dc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e10d1db4de26494c81b6bed2e25a2a50 in term 0.
I20260806 06:33:28.272862 26564 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee1526a6d25f408ebc63bb6e6a6f0cbf" candidate_uuid: "e10d1db4de26494c81b6bed2e25a2a50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49a8501de59748449a52600db82ef49f" is_pre_election: true
I20260806 06:33:28.272913 26662 leader_election.cc:302] T ee1526a6d25f408ebc63bb6e6a6f0cbf P e10d1db4de26494c81b6bed2e25a2a50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62ff3045c51d4c7aacdff1226a654dc2, e10d1db4de26494c81b6bed2e25a2a50; no voters:
I20260806 06:33:28.273026 26564 raft_consensus.cc:2454] T ee1526a6d25f408ebc63bb6e6a6f0cbf P 49a8501de59748449a52600db82ef49f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e10d1db4de26494c81b6bed2e25a2a50 in term 0.
I20260806 06:33:28.273034 26884 raft_consensus.cc:2781] T ee1526a6d25f408ebc63bb6e6a6f0cbf P e10d1db4de26494c81b6bed2e25a2a50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:33:28.273074 26884 raft_consensus.cc:482] T ee1526a6d25f408ebc63bb6e6a6f0cbf P e10d1db4de26494c81b6bed2e25a2a50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:33:28.273094 26884 raft_consensus.cc:3037] T ee1526a6d25f408ebc63bb6e6a6f0cbf P e10d1db4de26494c81b6bed2e25a2a50 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:33:28.273777 26884 raft_consensus.cc:504] T ee1526a6d25f408ebc63bb6e6a6f0cbf P e10d1db4de26494c81b6bed2e25a2a50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62ff3045c51d4c7aacdff1226a654dc2" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36967 } } peers { permanent_uuid: "e10d1db4de26494c81b6bed2e25a2a50" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 34539 } } peers { permanent_uuid: "49a8501de59748449a52600db82ef49f" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 36295 } }
I20260806 06:33:28.273885 26884 leader_election.cc:288] T ee1526a6d25f408ebc63bb6e6a6f0cbf P e10d1db4de26494c81b6bed2e25a2a50 [CANDIDATE]: Term 1 election: Requested vote from peers 62ff3045c51d4c7aacdff1226a654dc2 (127.7.43.65:36967), 49a8501de59748449a52600db82ef49f (127.7.43.66:36295)
I20260806 06:33:28.274065 26372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee1526a6d25f408ebc63bb6e6a6f0cbf" candidate_uuid: "e10d1db4de26494c81b6bed2e25a2a50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62ff3045c51d4c7aacdff1226a654dc2"
I20260806 06:33:28.274088 26564 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee1526a6d25f408ebc63bb6e6a6f0cbf" candidate_uuid: "e10d1db4de26494c81b6bed2e25a2a50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49a8501de59748449a52600db82ef49f"
I20260806 06:33:28.274139 26372 raft_consensus.cc:3037] T ee1526a6d25f408ebc63bb6e6a6f0cbf P 62ff3045c51d4c7aacdff1226a654dc2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:33:28.274173 26564 raft_consensus.cc:3037] T ee1526a6d25f408ebc63bb6e6a6f0cbf P 49a8501de59748449a52600db82ef49f [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:33:28.274999 26372 raft_consensus.cc:2454] T ee1526a6d25f408ebc63bb6e6a6f0cbf P 62ff3045c51d4c7aacdff1226a654dc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e10d1db4de26494c81b6bed2e25a2a50 in term 1.
I20260806 06:33:28.275064 26564 raft_consensus.cc:2454] T ee1526a6d25f408ebc63bb6e6a6f0cbf P 49a8501de59748449a52600db82ef49f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e10d1db4de26494c81b6bed2e25a2a50 in term 1.
I20260806 06:33:28.275182 26662 leader_election.cc:302] T ee1526a6d25f408ebc63bb6e6a6f0cbf P e10d1db4de26494c81b6bed2e25a2a50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62ff3045c51d4c7aacdff1226a654dc2, e10d1db4de26494c81b6bed2e25a2a50; no voters:
I20260806 06:33:28.275265 26884 raft_consensus.cc:2781] T ee1526a6d25f408ebc63bb6e6a6f0cbf P e10d1db4de26494c81b6bed2e25a2a50 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:33:28.275400 26884 raft_consensus.cc:686] T ee1526a6d25f408ebc63bb6e6a6f0cbf P e10d1db4de26494c81b6bed2e25a2a50 [term 1 LEADER]: Becoming Leader. State: Replica: e10d1db4de26494c81b6bed2e25a2a50, State: Running, Role: LEADER
I20260806 06:33:28.275489 26884 consensus_queue.cc:237] T ee1526a6d25f408ebc63bb6e6a6f0cbf P e10d1db4de26494c81b6bed2e25a2a50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62ff3045c51d4c7aacdff1226a654dc2" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36967 } } peers { permanent_uuid: "e10d1db4de26494c81b6bed2e25a2a50" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 34539 } } peers { permanent_uuid: "49a8501de59748449a52600db82ef49f" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 36295 } }
I20260806 06:33:28.276268 26138 catalog_manager.cc:5495] T ee1526a6d25f408ebc63bb6e6a6f0cbf P e10d1db4de26494c81b6bed2e25a2a50 reported cstate change: term changed from 0 to 1, leader changed from <none> to e10d1db4de26494c81b6bed2e25a2a50 (127.7.43.67). New cstate: current_term: 1 leader_uuid: "e10d1db4de26494c81b6bed2e25a2a50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62ff3045c51d4c7aacdff1226a654dc2" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e10d1db4de26494c81b6bed2e25a2a50" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 34539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49a8501de59748449a52600db82ef49f" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 36295 } health_report { overall_health: UNKNOWN } } }
I20260806 06:33:28.363344 26865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:28.360681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43290 (local address 127.7.43.67:34539)
0806 06:33:28.360726 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:28.360728 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:28.360737 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:33:28.360800 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:28.360800 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:28.360819 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:28.360875 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:33:28.361022 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:28.361491 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:28.362583 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:28.362808 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:28.362858 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:28.363035 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:33:28.363133 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:33:28.363186 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:28.363248 (+ 62us) server_negotiation.cc:299] Negotiation successful
0806 06:33:28.363267 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:33:28.365396 26372 raft_consensus.cc:1261] T ee1526a6d25f408ebc63bb6e6a6f0cbf P 62ff3045c51d4c7aacdff1226a654dc2 [term 1 FOLLOWER]: Refusing update from remote peer e10d1db4de26494c81b6bed2e25a2a50: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:33:28.365396 26564 raft_consensus.cc:1261] T ee1526a6d25f408ebc63bb6e6a6f0cbf P 49a8501de59748449a52600db82ef49f [term 1 FOLLOWER]: Refusing update from remote peer e10d1db4de26494c81b6bed2e25a2a50: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:33:28.365736 26884 consensus_queue.cc:1035] T ee1526a6d25f408ebc63bb6e6a6f0cbf P e10d1db4de26494c81b6bed2e25a2a50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49a8501de59748449a52600db82ef49f" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 36295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:33:28.365888 26904 consensus_queue.cc:1035] T ee1526a6d25f408ebc63bb6e6a6f0cbf P e10d1db4de26494c81b6bed2e25a2a50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62ff3045c51d4c7aacdff1226a654dc2" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 36967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:33:28.368304 26908 mvcc.cc:205] Tried to move back new op lower bound from 7315447842261577728 to 7315447841896800256. Current Snapshot: MvccSnapshot[applied={T|T < 7315447842261577728}]
I20260806 06:33:28.370018 26912 mvcc.cc:205] Tried to move back new op lower bound from 7315447842261577728 to 7315447841896800256. Current Snapshot: MvccSnapshot[applied={T|T < 7315447842261577728 or (T in {7315447842261577728})}]
I20260806 06:33:28.370573 26138 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48878:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:33:28.370663 26138 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48878:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:33:28.371431 26138 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 901f3fa5cdb240308ecdd5c870a2b433: Sending DeleteTablet for 3 replicas of tablet ee1526a6d25f408ebc63bb6e6a6f0cbf
I20260806 06:33:28.371789 26724 tablet_service.cc:1514] Processing DeleteTablet for tablet ee1526a6d25f408ebc63bb6e6a6f0cbf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:33:28 UTC) from {username='kudu', principal='kudu/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:43286
I20260806 06:33:28.371934 26340 tablet_service.cc:1514] Processing DeleteTablet for tablet ee1526a6d25f408ebc63bb6e6a6f0cbf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:33:28 UTC) from {username='kudu', principal='kudu/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:53728
I20260806 06:33:28.372182 26921 tablet_replica.cc:330] T ee1526a6d25f408ebc63bb6e6a6f0cbf P 62ff3045c51d4c7aacdff1226a654dc2: stopping tablet replica
I20260806 06:33:28.372207 26534 tablet_service.cc:1514] Processing DeleteTablet for tablet ee1526a6d25f408ebc63bb6e6a6f0cbf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:33:28 UTC) from {username='kudu', principal='kudu/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:60902
I20260806 06:33:28.372321 26921 raft_consensus.cc:2229] T ee1526a6d25f408ebc63bb6e6a6f0cbf P 62ff3045c51d4c7aacdff1226a654dc2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:33:28.372419 26921 raft_consensus.cc:2258] T ee1526a6d25f408ebc63bb6e6a6f0cbf P 62ff3045c51d4c7aacdff1226a654dc2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:33:28.372789 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 26199
I20260806 06:33:28.372926 26921 ts_tablet_manager.cc:1905] T ee1526a6d25f408ebc63bb6e6a6f0cbf P 62ff3045c51d4c7aacdff1226a654dc2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:33:28.378789 26922 tablet_replica.cc:330] T ee1526a6d25f408ebc63bb6e6a6f0cbf P 49a8501de59748449a52600db82ef49f: stopping tablet replica
I20260806 06:33:28.378903 26922 raft_consensus.cc:2229] T ee1526a6d25f408ebc63bb6e6a6f0cbf P 49a8501de59748449a52600db82ef49f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:33:28.378999 26922 raft_consensus.cc:2258] T ee1526a6d25f408ebc63bb6e6a6f0cbf P 49a8501de59748449a52600db82ef49f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:33:28.379154 26920 tablet_replica.cc:330] T ee1526a6d25f408ebc63bb6e6a6f0cbf P e10d1db4de26494c81b6bed2e25a2a50: stopping tablet replica
I20260806 06:33:28.379240 26920 raft_consensus.cc:2229] T ee1526a6d25f408ebc63bb6e6a6f0cbf P e10d1db4de26494c81b6bed2e25a2a50 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:33:28.379346 26920 raft_consensus.cc:2258] T ee1526a6d25f408ebc63bb6e6a6f0cbf P e10d1db4de26494c81b6bed2e25a2a50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:33:28.379601 26922 ts_tablet_manager.cc:1905] T ee1526a6d25f408ebc63bb6e6a6f0cbf P 49a8501de59748449a52600db82ef49f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:33:28.381487 26922 ts_tablet_manager.cc:1918] T ee1526a6d25f408ebc63bb6e6a6f0cbf P 49a8501de59748449a52600db82ef49f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260806 06:33:28.386525 26118 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.43.65:36967 (error 108)
I20260806 06:33:28.386732 26922 log.cc:1196] T ee1526a6d25f408ebc63bb6e6a6f0cbf P 49a8501de59748449a52600db82ef49f: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-1/wal/wals/ee1526a6d25f408ebc63bb6e6a6f0cbf
I20260806 06:33:28.386951 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 26444
I20260806 06:33:28.387007 26922 ts_tablet_manager.cc:1939] T ee1526a6d25f408ebc63bb6e6a6f0cbf P 49a8501de59748449a52600db82ef49f: Deleting consensus metadata
I20260806 06:33:28.391826 26920 ts_tablet_manager.cc:1905] T ee1526a6d25f408ebc63bb6e6a6f0cbf P e10d1db4de26494c81b6bed2e25a2a50: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:33:28.393322 26920 ts_tablet_manager.cc:1918] T ee1526a6d25f408ebc63bb6e6a6f0cbf P e10d1db4de26494c81b6bed2e25a2a50: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:33:28.393384 26920 log.cc:1196] T ee1526a6d25f408ebc63bb6e6a6f0cbf P e10d1db4de26494c81b6bed2e25a2a50: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785997861828271-7341-0/minicluster-data/ts-2/wal/wals/ee1526a6d25f408ebc63bb6e6a6f0cbf
I20260806 06:33:28.393617 26920 ts_tablet_manager.cc:1939] T ee1526a6d25f408ebc63bb6e6a6f0cbf P e10d1db4de26494c81b6bed2e25a2a50: Deleting consensus metadata
I20260806 06:33:28.394008 26119 catalog_manager.cc:4841] TS e10d1db4de26494c81b6bed2e25a2a50 (127.7.43.67:34539): tablet ee1526a6d25f408ebc63bb6e6a6f0cbf (table test-table [id=89060829aae345f4aa24e2d22f03bea6]) successfully deleted
I20260806 06:33:28.399595 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 26629
I20260806 06:33:28.406814 26863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:28.406594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:36967 (local address 127.0.0.1:53738)
0806 06:33:28.406745 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:28.406778 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.65:36967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
I20260806 06:33:28.406915 26863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:28.406661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.66:36295 (local address 127.0.0.1:60914)
0806 06:33:28.406888 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:28.406903 (+ 15us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.66:36295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219}
W20260806 06:33:28.407043 26118 catalog_manager.cc:4584] TS 62ff3045c51d4c7aacdff1226a654dc2 (127.7.43.65:36967): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ee1526a6d25f408ebc63bb6e6a6f0cbf: Network error: Client connection negotiation failed: client connection to 127.7.43.65:36967: connect: Connection refused (error 111)
I20260806 06:33:28.407191 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 26086
I20260806 06:33:28.421586 7341 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 06:33:28.765066 7341 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 06:33:28.765151 7341 mini_ranger.cc:66] Stopping Ranger...
I20260806 06:33:29.108558 7341 mini_ranger.cc:68] Stopped Ranger
2026-08-06 06:33:29.108 UTC [24753] LOG: received smart shutdown request
2026-08-06 06:33:29.112 UTC [24753] LOG: background worker "logical replication launcher" (PID 24763) exited with exit code 1
2026-08-06 06:33:29.112 UTC [24757] LOG: shutting down
2026-08-06 06:33:29.240 UTC [24753] LOG: database system is shut down
2026-08-06T06:33:29Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (46068 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:33:29 dist-test-slave-6zbq krb5kdc[26935](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:33:29 dist-test-slave-6zbq krb5kdc[26935](info): set up 2 sockets
Aug 06 06:33:29 dist-test-slave-6zbq krb5kdc[26935](info): commencing operation
krb5kdc: starting...
W20260806 06:33:30.449859 7341 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Aug 06 06:33:30 dist-test-slave-6zbq krb5kdc[26935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:33:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:33:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.43.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
I20260806 06:33:30.554514 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:34003
--webserver_interface=127.7.43.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:38967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.43.126:34003,127.7.43.125:44429,127.7.43.124:46439
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:33:30.634044 26953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:33:30.634229 26953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:33:30.634250 26953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:33:30.635341 26953 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:33:30.635380 26953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:33:30.635392 26953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:33:30.635403 26953 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:33:30.635419 26953 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:33:30.636799 26953 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:38967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.43.126:34003,127.7.43.125:44429,127.7.43.124:46439
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:34003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.43.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26953
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:33:30.636972 26953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:33:30.637138 26953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:33:30.641513 26964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:33:30.641573 26953 server_base.cc:1034] running on GCE node
W20260806 06:33:30.641494 26967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:33:30.641480 26963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:33:30.641994 26953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:33:30.642249 26953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:33:30.643383 26953 hybrid_clock.cc:648] HybridClock initialized: now 1785998010643378 us; error 37 us; skew 500 ppm
Aug 06 06:33:30 dist-test-slave-6zbq krb5kdc[26935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998010, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:33:30.649820 26953 init.cc:377] Logged in from keytab as kudu/127.7.43.126@KRBTEST.COM (short username kudu)
I20260806 06:33:30.650190 26953 webserver.cc:460] Webserver started at http://127.7.43.126:40585/ using document root <none> and password file <none>
I20260806 06:33:30.650413 26953 fs_manager.cc:359] Metadata directory not provided
I20260806 06:33:30.650462 26953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:33:30.650579 26953 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:33:30.651542 26953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/minicluster-data/master-0/data/instance:
uuid: "79d28096192a49218482b8f7ae5b25d9"
format_stamp: "Formatted at 2026-08-06 06:33:30 on dist-test-slave-6zbq"
server_key: "697e009a9f6b52f0c17296a2f226041d"
server_key_iv: "632fe6e0bebbbfecd3647443380f1ac0"
server_key_version: "encryptionkey@0"
I20260806 06:33:30.651868 26953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/minicluster-data/master-0/wal/instance:
uuid: "79d28096192a49218482b8f7ae5b25d9"
format_stamp: "Formatted at 2026-08-06 06:33:30 on dist-test-slave-6zbq"
server_key: "697e009a9f6b52f0c17296a2f226041d"
server_key_iv: "632fe6e0bebbbfecd3647443380f1ac0"
server_key_version: "encryptionkey@0"
I20260806 06:33:30.653077 26953 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:33:30.653724 26975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:30.653885 26953 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:33:30.653970 26953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "79d28096192a49218482b8f7ae5b25d9"
format_stamp: "Formatted at 2026-08-06 06:33:30 on dist-test-slave-6zbq"
server_key: "697e009a9f6b52f0c17296a2f226041d"
server_key_iv: "632fe6e0bebbbfecd3647443380f1ac0"
server_key_version: "encryptionkey@0"
I20260806 06:33:30.654026 26953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:33:30.678259 26953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:33:30.680318 26953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:33:30.680445 26953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:33:30.684188 26953 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.126:34003
I20260806 06:33:30.684230 27050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.126:34003 every 8 connection(s)
I20260806 06:33:30.684542 26953 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:33:30.685446 27051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:33:30.686995 27051 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.43.126" port: 34003 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:33:30.688514 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 26953
I20260806 06:33:30.688627 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/minicluster-data/master-0/wal/instance
I20260806 06:33:30.688819 7341 external_mini_cluster.cc:1428] Setting key 43542ab0b54178daeb58bc88d80c2e37
Aug 06 06:33:30 dist-test-slave-6zbq krb5kdc[26935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998010, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.126@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:33:30.696821 27053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:30.687623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:34003 (local address 127.0.0.1:51746)
0806 06:33:30.687767 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:30.687809 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 06:33:30.688264 (+ 455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:33:30.688532 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:33:30.688534 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:33:30.688808 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:33:30.688990 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:30.688994 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:30.690272 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:30.690272 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:30.690982 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:30.690985 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:30.691611 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:30.691612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:30.691691 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:30.692078 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:33:30.692087 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:33:30.693582 (+ 1495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:33:30.695415 (+ 1833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:33:30.695422 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:33:30.695424 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:33:30.695606 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:33:30.695883 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:33:30.695884 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:33:30.695885 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:33:30.695977 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:33:30.696388 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:33:30.696394 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:33:30.696451 (+ 57us) client_negotiation.cc:769] Sending connection context
0806 06:33:30.696527 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 06:33:30.696667 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":62,"threads_started":1}
I20260806 06:33:30.696900 27054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:30.687672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51746 (local address 127.7.43.126:34003)
0806 06:33:30.687843 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:30.687846 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:30.687873 (+ 27us) server_negotiation.cc:407] Connection header received
0806 06:33:30.688407 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:30.688409 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:30.688431 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:30.688519 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:33:30.689075 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:30.690181 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:30.691061 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:30.691519 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:30.691629 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:30.693658 (+ 2029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:33:30.693665 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:33:30.693665 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:33:30.693671 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:33:30.695337 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:30.695682 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:30.695685 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:30.695685 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:30.695786 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:30.696046 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:30.696047 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:30.696047 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:30.696315 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:33:30.696400 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:30.696573 (+ 173us) server_negotiation.cc:299] Negotiation successful
0806 06:33:30.696759 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":89,"server-negotiator.queue_time_us":135,"thread_start_us":91,"threads_started":1}
I20260806 06:33:30.697654 27051 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.43.125" port: 44429 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.7.43.125@KRBTEST.COM; defaulting to no policy
I20260806 06:33:30.697984 27053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:30.697883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.125:44429 (local address 127.0.0.1:36530)
0806 06:33:30.697921 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:30.697966 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.125:44429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260806 06:33:30.698109 26981 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.43.125:44429: connect: Connection refused (error 111)
I20260806 06:33:30.698503 27053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:30.698418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.125:44429 (local address 127.0.0.1:36536)
0806 06:33:30.698456 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:30.698483 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.125:44429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260806 06:33:30.698604 27051 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.43.125:44429: Network error: Client connection negotiation failed: client connection to 127.7.43.125:44429: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.125.keytab.
Entry for principal kudu/127.7.43.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.125.keytab.
I20260806 06:33:30.712879 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.125:44429
--webserver_interface=127.7.43.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.125.keytab
--principal=kudu/127.7.43.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:38967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.43.126:34003,127.7.43.125:44429,127.7.43.124:46439
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:33:30.747815 27051 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.43.125" port: 44429 } attempt: 1
I20260806 06:33:30.748279 27053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:30.748161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.125:44429 (local address 127.0.0.1:36550)
0806 06:33:30.748219 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:30.748250 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.125:44429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 06:33:30.748692 27053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:30.748608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.125:44429 (local address 127.0.0.1:36556)
0806 06:33:30.748646 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:30.748673 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.125:44429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260806 06:33:30.748790 27051 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.43.125:44429: Network error: Client connection negotiation failed: client connection to 127.7.43.125:44429: connect: Connection refused (error 111)
W20260806 06:33:30.792627 27058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:33:30.792802 27058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:33:30.792832 27058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:33:30.793926 27058 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:33:30.793963 27058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:33:30.793982 27058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:33:30.793998 27058 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:33:30.794013 27058 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:33:30.795475 27058 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:38967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.43.126:34003,127.7.43.125:44429,127.7.43.124:46439
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.125.keytab
--principal=kudu/127.7.43.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.125:44429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.43.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27058
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:33:30.795698 27058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:33:30.795887 27058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:33:30.800386 27058 server_base.cc:1034] running on GCE node
W20260806 06:33:30.800343 27064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:33:30.800359 27065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:33:30.800323 27070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:33:30.800791 27058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:33:30.801055 27058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:33:30.802210 27058 hybrid_clock.cc:648] HybridClock initialized: now 1785998010802197 us; error 30 us; skew 500 ppm
Aug 06 06:33:30 dist-test-slave-6zbq krb5kdc[26935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998010, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:33:30.804519 27058 init.cc:377] Logged in from keytab as kudu/127.7.43.125@KRBTEST.COM (short username kudu)
I20260806 06:33:30.804899 27058 webserver.cc:460] Webserver started at http://127.7.43.125:35047/ using document root <none> and password file <none>
I20260806 06:33:30.805102 27058 fs_manager.cc:359] Metadata directory not provided
I20260806 06:33:30.805145 27058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:33:30.805262 27058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:33:30.806202 27058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/minicluster-data/master-1/data/instance:
uuid: "0cd57d823dae4f1e8a954ac15d7a76fc"
format_stamp: "Formatted at 2026-08-06 06:33:30 on dist-test-slave-6zbq"
server_key: "a2d5b68113944f4748829f34cbe4cd9c"
server_key_iv: "2f9da69189742b093546d07fd74ffdd3"
server_key_version: "encryptionkey@0"
I20260806 06:33:30.806525 27058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/minicluster-data/master-1/wal/instance:
uuid: "0cd57d823dae4f1e8a954ac15d7a76fc"
format_stamp: "Formatted at 2026-08-06 06:33:30 on dist-test-slave-6zbq"
server_key: "a2d5b68113944f4748829f34cbe4cd9c"
server_key_iv: "2f9da69189742b093546d07fd74ffdd3"
server_key_version: "encryptionkey@0"
I20260806 06:33:30.807659 27058 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:33:30.808274 27078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:30.808418 27058 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:33:30.808498 27058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/minicluster-data/master-1/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/minicluster-data/master-1/wal
uuid: "0cd57d823dae4f1e8a954ac15d7a76fc"
format_stamp: "Formatted at 2026-08-06 06:33:30 on dist-test-slave-6zbq"
server_key: "a2d5b68113944f4748829f34cbe4cd9c"
server_key_iv: "2f9da69189742b093546d07fd74ffdd3"
server_key_version: "encryptionkey@0"
I20260806 06:33:30.808557 27058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:33:30.816165 27058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:33:30.816942 27051 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.43.125" port: 44429 } attempt: 2
I20260806 06:33:30.817260 27053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:30.817164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.125:44429 (local address 127.0.0.1:36560)
0806 06:33:30.817210 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:30.817236 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.125:44429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 06:33:30.817569 27053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:30.817489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.125:44429 (local address 127.0.0.1:36574)
0806 06:33:30.817528 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:30.817551 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.125:44429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260806 06:33:30.817652 27051 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.43.125:44429: Network error: Client connection negotiation failed: client connection to 127.7.43.125:44429: connect: Connection refused (error 111)
I20260806 06:33:30.818166 27058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:33:30.818269 27058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:33:30.821805 27058 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.125:44429
I20260806 06:33:30.821846 27149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.125:44429 every 8 connection(s)
I20260806 06:33:30.822113 27058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/minicluster-data/master-1/data/info.pb
I20260806 06:33:30.822911 27150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:33:30.824261 27150 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.43.126" port: 34003 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:33:30.827085 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 27058
I20260806 06:33:30.827203 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/minicluster-data/master-1/wal/instance
I20260806 06:33:30.827385 7341 external_mini_cluster.cc:1428] Setting key 88ff9cab39be656d62a8b51ee1cee7b6
Aug 06 06:33:30 dist-test-slave-6zbq krb5kdc[26935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998010, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.125@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:33:30.832487 27054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:30.824849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51754 (local address 127.7.43.126:34003)
0806 06:33:30.824899 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:30.824901 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:30.825077 (+ 176us) server_negotiation.cc:407] Connection header received
0806 06:33:30.825519 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:30.825520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:30.825532 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:30.825581 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:33:30.826116 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:30.826634 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:30.827645 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:30.827848 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:30.827902 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:30.829855 (+ 1953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:33:30.829856 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:33:30.829857 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:33:30.829863 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:33:30.831189 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:30.831577 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:30.831578 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:30.831578 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:30.831646 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:30.831909 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:30.831910 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:30.831910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:30.832039 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:33:30.832100 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:30.832315 (+ 215us) server_negotiation.cc:299] Negotiation successful
0806 06:33:30.832400 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 06:33:30.832528 27154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:30.824826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:34003 (local address 127.0.0.1:51754)
0806 06:33:30.824974 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:30.825006 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 06:33:30.825431 (+ 425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:33:30.825638 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:33:30.825641 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:33:30.825859 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:33:30.826031 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:30.826035 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:30.826730 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:30.826731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:30.827561 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:30.827564 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:30.827915 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:30.827916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:30.828041 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:30.828427 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:33:30.828435 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:33:30.829763 (+ 1328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:33:30.831292 (+ 1529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:33:30.831298 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:33:30.831299 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:33:30.831503 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:33:30.831727 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:33:30.831728 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:33:30.831729 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:33:30.831848 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:33:30.832098 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:33:30.832101 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:33:30.832199 (+ 98us) client_negotiation.cc:769] Sending connection context
0806 06:33:30.832277 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 06:33:30.832388 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":73,"threads_started":1}
I20260806 06:33:30.833048 27150 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.43.125" port: 44429 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.7.43.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.124@KRBTEST.COM" created.
Aug 06 06:33:30 dist-test-slave-6zbq krb5kdc[26935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998010, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.125@KRBTEST.COM for kudu/127.7.43.125@KRBTEST.COM
I20260806 06:33:30.841279 27154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:30.833283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.125:44429 (local address 127.0.0.1:36578)
0806 06:33:30.833329 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:30.833341 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:33:30.833382 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:33:30.833585 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:33:30.833586 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:33:30.833732 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:33:30.833787 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:30.833788 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:30.834607 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:30.834608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:30.835359 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:30.835362 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:30.835760 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:30.835761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:30.835836 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:30.836144 (+ 308us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:33:30.836149 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:33:30.838870 (+ 2721us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:33:30.840341 (+ 1471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:33:30.840342 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:33:30.840343 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:33:30.840536 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:33:30.840745 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:33:30.840746 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:33:30.840746 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:33:30.840794 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:33:30.841058 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:33:30.841059 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:33:30.841090 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 06:33:30.841142 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 06:33:30.841213 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 06:33:30.841389 27157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:30.833320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36578 (local address 127.7.43.125:44429)
0806 06:33:30.833451 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:30.833456 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:30.833469 (+ 13us) server_negotiation.cc:407] Connection header received
0806 06:33:30.833487 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:30.833488 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:30.833498 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:30.833566 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:33:30.833853 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:30.834533 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:30.835432 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:30.835697 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:30.835744 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:30.838946 (+ 3202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:33:30.838952 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:33:30.838952 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:33:30.838958 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:33:30.840258 (+ 1300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:30.840622 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:30.840624 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:30.840625 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:30.840692 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:30.840843 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:30.840844 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:30.840844 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:30.841000 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:33:30.841072 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:30.841177 (+ 105us) server_negotiation.cc:299] Negotiation successful
0806 06:33:30.841276 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":105,"thread_start_us":72,"threads_started":1}
I20260806 06:33:30.841918 27150 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.43.124" port: 46439 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:33:30.842208 27154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:30.842111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.124:46439 (local address 127.0.0.1:33192)
0806 06:33:30.842151 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:30.842190 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.124:46439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260806 06:33:30.842368 27079 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.43.124:46439: connect: Connection refused (error 111)
I20260806 06:33:30.842742 27154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:30.842665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.124:46439 (local address 127.0.0.1:33198)
0806 06:33:30.842704 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:30.842726 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.124:46439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260806 06:33:30.842839 27150 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.43.124:46439: Network error: Client connection negotiation failed: client connection to 127.7.43.124:46439: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.124.keytab.
Entry for principal kudu/127.7.43.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.124.keytab.
I20260806 06:33:30.850886 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.124:46439
--webserver_interface=127.7.43.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.124.keytab
--principal=kudu/127.7.43.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:38967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.43.126:34003,127.7.43.125:44429,127.7.43.124:46439
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:33:30.892059 27150 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.43.124" port: 46439 } attempt: 1
I20260806 06:33:30.892441 27154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:30.892338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.124:46439 (local address 127.0.0.1:33204)
0806 06:33:30.892388 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:30.892414 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.124:46439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 06:33:30.892776 27154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:30.892708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.124:46439 (local address 127.0.0.1:33206)
0806 06:33:30.892739 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:30.892761 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.124:46439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260806 06:33:30.892861 27150 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.43.124:46439: Network error: Client connection negotiation failed: client connection to 127.7.43.124:46439: connect: Connection refused (error 111)
I20260806 06:33:30.908782 27051 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.43.125" port: 44429 } attempt: 3
Aug 06 06:33:30 dist-test-slave-6zbq krb5kdc[26935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998010, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.126@KRBTEST.COM for kudu/127.7.43.125@KRBTEST.COM
I20260806 06:33:30.915360 27053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:30.909040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.125:44429 (local address 127.0.0.1:36588)
0806 06:33:30.909092 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:30.909105 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:33:30.909152 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:33:30.909285 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:33:30.909286 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:33:30.909423 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:33:30.909492 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:30.909494 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:30.910431 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:30.910431 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:30.910943 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:30.910945 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:30.911382 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:30.911383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:30.911442 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:30.911846 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:33:30.911851 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:33:30.912952 (+ 1101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:33:30.914532 (+ 1580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:33:30.914533 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:33:30.914533 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:33:30.914686 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:33:30.914866 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:33:30.914867 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:33:30.914867 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:33:30.914912 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:33:30.915142 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:33:30.915142 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:33:30.915178 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 06:33:30.915220 (+ 42us) client_negotiation.cc:240] Negotiation successful
0806 06:33:30.915298 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 06:33:30.915411 27157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:30.909112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36588 (local address 127.7.43.125:44429)
0806 06:33:30.909161 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:30.909163 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:30.909173 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:33:30.909199 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:30.909200 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:30.909209 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:30.909274 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:33:30.909559 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:30.910330 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:30.911011 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:30.911314 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:30.911388 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:30.913033 (+ 1645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:33:30.913034 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:33:30.913034 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:33:30.913043 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:33:30.914461 (+ 1418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:30.914747 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:30.914748 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:30.914749 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:30.914814 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:30.914964 (+ 150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:30.914965 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:30.914966 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:30.915088 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:33:30.915149 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:30.915247 (+ 98us) server_negotiation.cc:299] Negotiation successful
0806 06:33:30.915325 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 06:33:30.915686 27051 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.43.124" port: 46439 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:33:30.915964 27053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:30.915872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.124:46439 (local address 127.0.0.1:33216)
0806 06:33:30.915914 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:30.915940 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.124:46439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 06:33:30.916273 27053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:30.916197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.124:46439 (local address 127.0.0.1:33222)
0806 06:33:30.916231 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:30.916253 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.124:46439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260806 06:33:30.916350 27051 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.43.124:46439: Network error: Client connection negotiation failed: client connection to 127.7.43.124:46439: connect: Connection refused (error 111)
W20260806 06:33:30.930153 27163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:33:30.930328 27163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:33:30.930377 27163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:33:30.931470 27163 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:33:30.931506 27163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:33:30.931519 27163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:33:30.931541 27163 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:33:30.931563 27163 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:33:30.932960 27163 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:38967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.43.126:34003,127.7.43.125:44429,127.7.43.124:46439
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.124.keytab
--principal=kudu/127.7.43.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.124:46439
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.43.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27163
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:33:30.933159 27163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:33:30.933354 27163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:33:30.937597 27169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:33:30.937721 27163 server_base.cc:1034] running on GCE node
W20260806 06:33:30.937618 27171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:33:30.937649 27176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:33:30.938023 27163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:33:30.938300 27163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:33:30.939456 27163 hybrid_clock.cc:648] HybridClock initialized: now 1785998010939444 us; error 26 us; skew 500 ppm
Aug 06 06:33:30 dist-test-slave-6zbq krb5kdc[26935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998010, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:33:30.941622 27163 init.cc:377] Logged in from keytab as kudu/127.7.43.124@KRBTEST.COM (short username kudu)
I20260806 06:33:30.942010 27163 webserver.cc:460] Webserver started at http://127.7.43.124:42047/ using document root <none> and password file <none>
I20260806 06:33:30.942292 27163 fs_manager.cc:359] Metadata directory not provided
I20260806 06:33:30.942384 27163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:33:30.942513 27163 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:33:30.943480 27163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/minicluster-data/master-2/data/instance:
uuid: "3595b6a0d71242bb8b4a9f4781eb361e"
format_stamp: "Formatted at 2026-08-06 06:33:30 on dist-test-slave-6zbq"
server_key: "548a64bfeee4b970a32448169f73e468"
server_key_iv: "bef2677848a85e3f81cb101f70c827fb"
server_key_version: "encryptionkey@0"
I20260806 06:33:30.943787 27163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/minicluster-data/master-2/wal/instance:
uuid: "3595b6a0d71242bb8b4a9f4781eb361e"
format_stamp: "Formatted at 2026-08-06 06:33:30 on dist-test-slave-6zbq"
server_key: "548a64bfeee4b970a32448169f73e468"
server_key_iv: "bef2677848a85e3f81cb101f70c827fb"
server_key_version: "encryptionkey@0"
I20260806 06:33:30.944944 27163 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:33:30.945533 27186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:30.945678 27163 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:30.945758 27163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/minicluster-data/master-2/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/minicluster-data/master-2/wal
uuid: "3595b6a0d71242bb8b4a9f4781eb361e"
format_stamp: "Formatted at 2026-08-06 06:33:30 on dist-test-slave-6zbq"
server_key: "548a64bfeee4b970a32448169f73e468"
server_key_iv: "bef2677848a85e3f81cb101f70c827fb"
server_key_version: "encryptionkey@0"
I20260806 06:33:30.945816 27163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:33:30.947465 27051 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.43.124" port: 46439 } attempt: 1
I20260806 06:33:30.947747 27053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:30.947661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.124:46439 (local address 127.0.0.1:33234)
0806 06:33:30.947702 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:30.947726 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.124:46439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260806 06:33:30.948086 27053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:30.948006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.124:46439 (local address 127.0.0.1:33240)
0806 06:33:30.948046 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:30.948069 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.124:46439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260806 06:33:30.948168 27051 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.43.124:46439: Network error: Client connection negotiation failed: client connection to 127.7.43.124:46439: connect: Connection refused (error 111)
I20260806 06:33:30.961010 27150 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.43.124" port: 46439 } attempt: 2
I20260806 06:33:30.961347 27154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:30.961252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.124:46439 (local address 127.0.0.1:33248)
0806 06:33:30.961296 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:30.961326 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.124:46439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260806 06:33:30.961633 27154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:30.961562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.124:46439 (local address 127.0.0.1:33252)
0806 06:33:30.961595 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:30.961617 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.124:46439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260806 06:33:30.961709 27150 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.43.124:46439: Network error: Client connection negotiation failed: client connection to 127.7.43.124:46439: connect: Connection refused (error 111)
I20260806 06:33:30.967358 27163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:33:30.968900 27163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:33:30.969022 27163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:33:30.972664 27163 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.124:46439
I20260806 06:33:30.972721 27261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.124:46439 every 8 connection(s)
I20260806 06:33:30.972990 27163 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/minicluster-data/master-2/data/info.pb
I20260806 06:33:30.973735 27262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:33:30.975005 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 27163
I20260806 06:33:30.975126 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785997861828271-7341-0/minicluster-data/master-2/wal/instance
I20260806 06:33:30.975322 7341 external_mini_cluster.cc:1428] Setting key 7ea04e95c4ce935a890e623cb559ce42
I20260806 06:33:30.975378 27262 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.43.126" port: 34003 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:33:30 dist-test-slave-6zbq krb5kdc[26935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998010, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.124@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
Aug 06 06:33:30 dist-test-slave-6zbq krb5kdc[26935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.43.124@KRBTEST.COM
I20260806 06:33:30.983312 27054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:30.976134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51756 (local address 127.7.43.126:34003)
0806 06:33:30.976191 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:30.976193 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:30.976332 (+ 139us) server_negotiation.cc:407] Connection header received
0806 06:33:30.976518 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:30.976519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:30.976531 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:30.976579 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:33:30.977077 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:30.977542 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:30.978391 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:30.978582 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:30.978639 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:30.980661 (+ 2022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:33:30.980663 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:33:30.980663 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:33:30.980670 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:33:30.981897 (+ 1227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:30.982374 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:30.982375 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:30.982376 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:30.982437 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:30.982657 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:30.982659 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:30.982659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:30.982795 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:33:30.982867 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:30.983140 (+ 273us) server_negotiation.cc:299] Negotiation successful
0806 06:33:30.983228 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 06:33:30.983434 27268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:30.976088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:34003 (local address 127.0.0.1:51756)
0806 06:33:30.976234 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:30.976265 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 06:33:30.976441 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:33:30.976597 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:33:30.976599 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:33:30.976837 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:33:30.976992 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:30.976996 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:30.977612 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:30.977614 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:30.978296 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:30.978298 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:30.978651 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:30.978652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:30.978768 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:30.979168 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:33:30.979178 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:33:30.980582 (+ 1404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:33:30.981963 (+ 1381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:33:30.981970 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:33:30.981973 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:33:30.982247 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:33:30.982501 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:33:30.982501 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:33:30.982501 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:33:30.982598 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:33:30.982854 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:33:30.982858 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:33:30.983002 (+ 144us) client_negotiation.cc:769] Sending connection context
0806 06:33:30.983110 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 06:33:30.983240 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":76,"threads_started":1}
I20260806 06:33:30.983987 27262 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.43.125" port: 44429 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:33:30.984182 27267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:30.975738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33262 (local address 127.7.43.124:46439)
0806 06:33:30.975897 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:30.975906 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:30.975936 (+ 30us) server_negotiation.cc:407] Connection header received
0806 06:33:30.976422 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:30.976429 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:30.976449 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:30.976528 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:33:30.977135 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:30.977892 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:30.978639 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:30.978924 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:30.978982 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:30.981190 (+ 2208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:33:30.981198 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:33:30.981199 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:33:30.981206 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:33:30.982822 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:30.983176 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:30.983178 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:30.983179 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:30.983237 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:30.983502 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:30.983502 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:30.983503 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:30.983661 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:33:30.983811 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:30.983903 (+ 92us) server_negotiation.cc:299] Negotiation successful
0806 06:33:30.983968 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":65,"threads_started":1}
Aug 06 06:33:30 dist-test-slave-6zbq krb5kdc[26935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998010, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.124@KRBTEST.COM for kudu/127.7.43.125@KRBTEST.COM
Aug 06 06:33:30 dist-test-slave-6zbq krb5kdc[26935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:33:30.990563 27268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:30.984225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.125:44429 (local address 127.0.0.1:36592)
0806 06:33:30.984265 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:30.984274 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:33:30.984308 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:33:30.984423 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:33:30.984423 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:33:30.984538 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:33:30.984581 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:30.984582 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:30.985375 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:30.985376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:30.986103 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:30.986105 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:30.986567 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:30.986568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:30.986615 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:30.986936 (+ 321us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:33:30.986940 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:33:30.988116 (+ 1176us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:33:30.989655 (+ 1539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:33:30.989656 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:33:30.989656 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:33:30.989848 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:33:30.990036 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:33:30.990037 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:33:30.990037 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:33:30.990090 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:33:30.990359 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:33:30.990359 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:33:30.990397 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 06:33:30.990445 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 06:33:30.990508 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 06:33:30.990628 27157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:30.984290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36592 (local address 127.7.43.125:44429)
0806 06:33:30.984325 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:30.984327 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:30.984336 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:33:30.984356 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:30.984357 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:30.984366 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:30.984421 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:33:30.984653 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:30.985305 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:30.986198 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:30.986506 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:30.986571 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:30.988213 (+ 1642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:33:30.988214 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:33:30.988214 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:33:30.988222 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:33:30.989569 (+ 1347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:30.989923 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:30.989924 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:30.989924 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:30.989981 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:30.990151 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:30.990152 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:30.990152 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:30.990277 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:33:30.990368 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:30.990474 (+ 106us) server_negotiation.cc:299] Negotiation successful
0806 06:33:30.990540 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:33:30.990906 27262 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.43.124" port: 46439 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:33:30.991768 27054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:30.984818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51762 (local address 127.7.43.126:34003)
0806 06:33:30.984852 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:30.984853 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:30.984861 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:33:30.984911 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:30.984912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:30.984919 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:30.984954 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:33:30.985437 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:30.986120 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:30.986903 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:30.987092 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:30.987142 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:30.989101 (+ 1959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:33:30.989102 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:33:30.989103 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:33:30.989110 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:33:30.990629 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:30.990990 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:30.990991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:30.990991 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:30.991054 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:30.991277 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:30.991278 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:30.991278 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:30.991405 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:33:30.991559 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:30.991606 (+ 47us) server_negotiation.cc:299] Negotiation successful
0806 06:33:30.991683 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
Aug 06 06:33:30 dist-test-slave-6zbq krb5kdc[26935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998010, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.124@KRBTEST.COM for kudu/127.7.43.124@KRBTEST.COM
Aug 06 06:33:30 dist-test-slave-6zbq krb5kdc[26935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.43.125@KRBTEST.COM
I20260806 06:33:30.997224 27268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:30.991117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.124:46439 (local address 127.0.0.1:33274)
0806 06:33:30.991155 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:30.991163 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:33:30.991203 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:33:30.991312 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:33:30.991313 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:33:30.991413 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:33:30.991440 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:30.991441 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:30.992297 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:30.992298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:30.992992 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:30.992994 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:30.993296 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:30.993297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:30.993340 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:30.993677 (+ 337us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:33:30.993682 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:33:30.994951 (+ 1269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:33:30.996303 (+ 1352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:33:30.996304 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:33:30.996304 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:33:30.996496 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:33:30.996679 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:33:30.996680 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:33:30.996680 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:33:30.996729 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:33:30.996938 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:33:30.996938 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:33:30.996974 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 06:33:30.997038 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 06:33:30.997140 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 06:33:30.997407 27267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:30.991194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33274 (local address 127.7.43.124:46439)
0806 06:33:30.991225 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:30.991227 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:30.991233 (+ 6us) server_negotiation.cc:407] Connection header received
0806 06:33:30.991250 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:30.991251 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:30.991258 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:30.991311 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:33:30.991508 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:30.992117 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:30.993056 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:30.993244 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:30.993292 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:30.995014 (+ 1722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:33:30.995015 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:33:30.995015 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:33:30.995022 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:33:30.996220 (+ 1198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:30.996572 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:30.996573 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:30.996574 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:30.996630 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:30.996776 (+ 146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:30.996776 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:30.996777 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:30.996869 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:33:30.996920 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:30.997082 (+ 162us) server_negotiation.cc:299] Negotiation successful
0806 06:33:30.997216 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":15}
I20260806 06:33:30.998631 27262 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3595b6a0d71242bb8b4a9f4781eb361e: Bootstrap starting.
I20260806 06:33:30.999135 27157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:30.992293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36600 (local address 127.7.43.125:44429)
0806 06:33:30.992325 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:30.992327 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:30.992349 (+ 22us) server_negotiation.cc:407] Connection header received
0806 06:33:30.992368 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:30.992369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:30.992376 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:30.992408 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:33:30.992899 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:30.993430 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:30.994283 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:30.994585 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:30.994648 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:30.996823 (+ 2175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:33:30.996824 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:33:30.996825 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:33:30.996831 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:33:30.998166 (+ 1335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:30.998487 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:30.998488 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:30.998488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:30.998538 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:30.998699 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:30.998700 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:30.998700 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:30.998808 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:33:30.998963 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:30.998997 (+ 34us) server_negotiation.cc:299] Negotiation successful
0806 06:33:30.999060 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 06:33:30.999447 27262 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3595b6a0d71242bb8b4a9f4781eb361e: Neither blocks nor log segments found. Creating new log.
I20260806 06:33:30.999722 27262 log.cc:824] T 00000000000000000000000000000000 P 3595b6a0d71242bb8b4a9f4781eb361e: Log is configured to *not* fsync() on all Append() calls
I20260806 06:33:31.000473 27262 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3595b6a0d71242bb8b4a9f4781eb361e: No bootstrap required, opened a new log
I20260806 06:33:31.000700 7341 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:33:31.002045 27262 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3595b6a0d71242bb8b4a9f4781eb361e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79d28096192a49218482b8f7ae5b25d9" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 34003 } } peers { permanent_uuid: "0cd57d823dae4f1e8a954ac15d7a76fc" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 44429 } } peers { permanent_uuid: "3595b6a0d71242bb8b4a9f4781eb361e" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 46439 } }
I20260806 06:33:31.002272 27262 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3595b6a0d71242bb8b4a9f4781eb361e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:33:31.002331 27262 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3595b6a0d71242bb8b4a9f4781eb361e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3595b6a0d71242bb8b4a9f4781eb361e, State: Initialized, Role: FOLLOWER
I20260806 06:33:31.002489 27262 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3595b6a0d71242bb8b4a9f4781eb361e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79d28096192a49218482b8f7ae5b25d9" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 34003 } } peers { permanent_uuid: "0cd57d823dae4f1e8a954ac15d7a76fc" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 44429 } } peers { permanent_uuid: "3595b6a0d71242bb8b4a9f4781eb361e" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 46439 } }
I20260806 06:33:31.002883 27262 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3595b6a0d71242bb8b4a9f4781eb361e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:33:31.005342 27283 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3595b6a0d71242bb8b4a9f4781eb361e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79d28096192a49218482b8f7ae5b25d9" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 34003 } } peers { permanent_uuid: "0cd57d823dae4f1e8a954ac15d7a76fc" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 44429 } } peers { permanent_uuid: "3595b6a0d71242bb8b4a9f4781eb361e" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 46439 } } }
I20260806 06:33:31.005441 27283 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3595b6a0d71242bb8b4a9f4781eb361e [sys.catalog]: This master's current role is: FOLLOWER
W20260806 06:33:31.005944 27298 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3595b6a0d71242bb8b4a9f4781eb361e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:33:31.006034 27298 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:33:31.004815 27163 master_runner.cc:186] Error getting master registration for 127.7.43.126:34003: OK, instance_id { permanent_uuid: "79d28096192a49218482b8f7ae5b25d9" instance_seqno: 1785998010683131 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 06:33:31.008648 27163 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 06:33:31.009802 27267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:31.001983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33278 (local address 127.7.43.124:46439)
0806 06:33:31.002014 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:31.002016 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:31.002117 (+ 101us) server_negotiation.cc:407] Connection header received
0806 06:33:31.002202 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:31.002202 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:31.002210 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:31.002236 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:33:31.003033 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:31.003678 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:31.004782 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:31.005037 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:31.005078 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:31.007267 (+ 2189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:33:31.007268 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:33:31.007268 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:33:31.007274 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:33:31.008725 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:31.009036 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:31.009037 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:31.009038 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:31.009093 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:31.009324 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:31.009325 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:31.009326 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:31.009448 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:33:31.009526 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:31.009658 (+ 132us) server_negotiation.cc:299] Negotiation successful
0806 06:33:31.009722 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:33:31.010964 27157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:31.001865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36612 (local address 127.7.43.125:44429)
0806 06:33:31.001898 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:31.001899 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:31.002125 (+ 226us) server_negotiation.cc:407] Connection header received
0806 06:33:31.002188 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:31.002189 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:31.002198 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:31.002254 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:33:31.002938 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:31.003594 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:31.004696 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:31.004951 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:31.004985 (+ 34us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:31.006457 (+ 1472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:33:31.006458 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:33:31.006459 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:33:31.006465 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:33:31.008694 (+ 2229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:31.009405 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:31.009406 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:31.009406 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:31.009461 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:31.010322 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:31.010323 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:31.010323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:31.010462 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:33:31.010565 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:31.010834 (+ 269us) server_negotiation.cc:299] Negotiation successful
0806 06:33:31.010894 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 06:33:31.011554 27054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:31.001880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51766 (local address 127.7.43.126:34003)
0806 06:33:31.001913 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:31.001915 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:31.002594 (+ 679us) server_negotiation.cc:407] Connection header received
0806 06:33:31.002693 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:31.002694 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:31.002703 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:31.002732 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:33:31.003374 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:31.004029 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:31.004886 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:31.005179 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:31.005218 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:31.007271 (+ 2053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:33:31.007273 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:33:31.007273 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:33:31.007280 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:33:31.010113 (+ 2833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:31.010639 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:31.010640 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:31.010641 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:31.010701 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:31.011008 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:31.011009 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:31.011010 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:31.011149 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:33:31.011231 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:31.011399 (+ 168us) server_negotiation.cc:299] Negotiation successful
0806 06:33:31.011472 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 06:33:31.023298 27051 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.43.124" port: 46439 } attempt: 2
Aug 06 06:33:31 dist-test-slave-6zbq krb5kdc[26935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998010, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.126@KRBTEST.COM for kudu/127.7.43.124@KRBTEST.COM
I20260806 06:33:31.028613 27053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:31.023497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.124:46439 (local address 127.0.0.1:33294)
0806 06:33:31.023541 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:31.023550 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:33:31.023582 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:33:31.023701 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:33:31.023702 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:33:31.023814 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:33:31.023843 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:31.023844 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:31.024392 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:31.024393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:31.024842 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:31.024843 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:31.025164 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:31.025165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:31.025214 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:31.025519 (+ 305us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:33:31.025524 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:33:31.026525 (+ 1001us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:33:31.027787 (+ 1262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:33:31.027787 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:33:31.027788 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:33:31.027935 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:33:31.028088 (+ 153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:33:31.028089 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:33:31.028089 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:33:31.028152 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:33:31.028339 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:33:31.028340 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:33:31.028380 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 06:33:31.028440 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 06:33:31.028524 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260806 06:33:31.028607 27267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:31.023557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33294 (local address 127.7.43.124:46439)
0806 06:33:31.023605 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:31.023607 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:31.023615 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:33:31.023642 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:31.023642 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:31.023650 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:31.023692 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:33:31.023902 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:31.024328 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:31.024905 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:31.025105 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:31.025155 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:31.026595 (+ 1440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:33:31.026597 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:33:31.026597 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:33:31.026603 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:33:31.027715 (+ 1112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:31.027992 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:31.027993 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:31.027993 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:31.028038 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:31.028199 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:31.028200 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:31.028201 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:31.028288 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:33:31.028338 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:31.028467 (+ 129us) server_negotiation.cc:299] Negotiation successful
0806 06:33:31.028524 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 06:33:31.029917 27051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79d28096192a49218482b8f7ae5b25d9: Bootstrap starting.
I20260806 06:33:31.030473 27051 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 79d28096192a49218482b8f7ae5b25d9: Neither blocks nor log segments found. Creating new log.
I20260806 06:33:31.030721 27051 log.cc:824] T 00000000000000000000000000000000 P 79d28096192a49218482b8f7ae5b25d9: Log is configured to *not* fsync() on all Append() calls
I20260806 06:33:31.031386 27051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79d28096192a49218482b8f7ae5b25d9: No bootstrap required, opened a new log
I20260806 06:33:31.032706 27051 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79d28096192a49218482b8f7ae5b25d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79d28096192a49218482b8f7ae5b25d9" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 34003 } } peers { permanent_uuid: "0cd57d823dae4f1e8a954ac15d7a76fc" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 44429 } } peers { permanent_uuid: "3595b6a0d71242bb8b4a9f4781eb361e" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 46439 } }
I20260806 06:33:31.032805 27051 raft_consensus.cc:374] T 00000000000000000000000000000000 P 79d28096192a49218482b8f7ae5b25d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:33:31.032826 27051 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79d28096192a49218482b8f7ae5b25d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79d28096192a49218482b8f7ae5b25d9, State: Initialized, Role: FOLLOWER
I20260806 06:33:31.032930 27051 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79d28096192a49218482b8f7ae5b25d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79d28096192a49218482b8f7ae5b25d9" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 34003 } } peers { permanent_uuid: "0cd57d823dae4f1e8a954ac15d7a76fc" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 44429 } } peers { permanent_uuid: "3595b6a0d71242bb8b4a9f4781eb361e" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 46439 } }
I20260806 06:33:31.033182 27051 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79d28096192a49218482b8f7ae5b25d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:33:31.033171 27303 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79d28096192a49218482b8f7ae5b25d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79d28096192a49218482b8f7ae5b25d9" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 34003 } } peers { permanent_uuid: "0cd57d823dae4f1e8a954ac15d7a76fc" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 44429 } } peers { permanent_uuid: "3595b6a0d71242bb8b4a9f4781eb361e" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 46439 } } }
I20260806 06:33:31.033273 27303 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79d28096192a49218482b8f7ae5b25d9 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 06:33:31.034709 27326 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 79d28096192a49218482b8f7ae5b25d9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:33:31.034811 27326 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:33:31.035230 26953 master_runner.cc:186] Error getting master registration for 127.7.43.125:44429: OK, instance_id { permanent_uuid: "0cd57d823dae4f1e8a954ac15d7a76fc" instance_seqno: 1785998010820839 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 06:33:31.037954 26953 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 06:33:31.052829 27150 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.43.124" port: 46439 } attempt: 3
Aug 06 06:33:31 dist-test-slave-6zbq krb5kdc[26935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998010, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.125@KRBTEST.COM for kudu/127.7.43.124@KRBTEST.COM
I20260806 06:33:31.060225 27267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:31.053091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33304 (local address 127.7.43.124:46439)
0806 06:33:31.053124 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:31.053126 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:31.053134 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:33:31.053229 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:31.053230 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:31.053238 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:31.053295 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:33:31.053585 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:31.054310 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:31.055264 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:31.055553 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:31.055613 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:31.057698 (+ 2085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:33:31.057699 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:33:31.057700 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:33:31.057706 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:33:31.059186 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:31.059539 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:31.059540 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:31.059540 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:31.059593 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:31.059758 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:31.059759 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:31.059759 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:31.059869 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:33:31.059929 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:31.060063 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 06:33:31.060138 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:33:31.060230 27154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:31.053047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.124:46439 (local address 127.0.0.1:33304)
0806 06:33:31.053086 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:31.053095 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:33:31.053126 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:33:31.053305 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:33:31.053306 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:33:31.053458 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:33:31.053500 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:31.053502 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:31.054414 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:31.054415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:31.055161 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:31.055164 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:31.055618 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:31.055620 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:31.055685 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:31.056079 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:33:31.056084 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:33:31.057605 (+ 1521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:33:31.059267 (+ 1662us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:33:31.059268 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:33:31.059269 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:33:31.059471 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:33:31.059649 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:33:31.059650 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:33:31.059650 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:33:31.059706 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:33:31.059926 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:33:31.059926 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:33:31.059969 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 06:33:31.060037 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 06:33:31.060132 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260806 06:33:31.061862 27150 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cd57d823dae4f1e8a954ac15d7a76fc: Bootstrap starting.
I20260806 06:33:31.062551 27150 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0cd57d823dae4f1e8a954ac15d7a76fc: Neither blocks nor log segments found. Creating new log.
I20260806 06:33:31.062834 27150 log.cc:824] T 00000000000000000000000000000000 P 0cd57d823dae4f1e8a954ac15d7a76fc: Log is configured to *not* fsync() on all Append() calls
I20260806 06:33:31.063616 27150 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cd57d823dae4f1e8a954ac15d7a76fc: No bootstrap required, opened a new log
I20260806 06:33:31.065506 27150 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0cd57d823dae4f1e8a954ac15d7a76fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79d28096192a49218482b8f7ae5b25d9" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 34003 } } peers { permanent_uuid: "0cd57d823dae4f1e8a954ac15d7a76fc" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 44429 } } peers { permanent_uuid: "3595b6a0d71242bb8b4a9f4781eb361e" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 46439 } }
I20260806 06:33:31.065620 27150 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0cd57d823dae4f1e8a954ac15d7a76fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:33:31.065650 27150 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0cd57d823dae4f1e8a954ac15d7a76fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cd57d823dae4f1e8a954ac15d7a76fc, State: Initialized, Role: FOLLOWER
I20260806 06:33:31.065755 27150 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0cd57d823dae4f1e8a954ac15d7a76fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79d28096192a49218482b8f7ae5b25d9" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 34003 } } peers { permanent_uuid: "0cd57d823dae4f1e8a954ac15d7a76fc" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 44429 } } peers { permanent_uuid: "3595b6a0d71242bb8b4a9f4781eb361e" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 46439 } }
I20260806 06:33:31.066021 27150 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0cd57d823dae4f1e8a954ac15d7a76fc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:33:31.065990 27330 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cd57d823dae4f1e8a954ac15d7a76fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79d28096192a49218482b8f7ae5b25d9" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 34003 } } peers { permanent_uuid: "0cd57d823dae4f1e8a954ac15d7a76fc" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 44429 } } peers { permanent_uuid: "3595b6a0d71242bb8b4a9f4781eb361e" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 46439 } } }
I20260806 06:33:31.066171 27330 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cd57d823dae4f1e8a954ac15d7a76fc [sys.catalog]: This master's current role is: FOLLOWER
W20260806 06:33:31.067821 27343 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0cd57d823dae4f1e8a954ac15d7a76fc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:33:31.067920 27343 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260806 06:33:31.068781 27087 tablet.cc:2369] T 00000000000000000000000000000000 P 0cd57d823dae4f1e8a954ac15d7a76fc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:33:31.069615 27058 master_runner.cc:263] No leader master found from master 0cd57d823dae4f1e8a954ac15d7a76fc
I20260806 06:33:31.069659 27058 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 06:33:31.181182 26983 tablet.cc:2369] T 00000000000000000000000000000000 P 79d28096192a49218482b8f7ae5b25d9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 06:33:31.219609 27193 tablet.cc:2369] T 00000000000000000000000000000000 P 3595b6a0d71242bb8b4a9f4781eb361e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:33:31.373224 27303 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79d28096192a49218482b8f7ae5b25d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:33:31.373328 27303 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79d28096192a49218482b8f7ae5b25d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79d28096192a49218482b8f7ae5b25d9" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 34003 } } peers { permanent_uuid: "0cd57d823dae4f1e8a954ac15d7a76fc" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 44429 } } peers { permanent_uuid: "3595b6a0d71242bb8b4a9f4781eb361e" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 46439 } }
I20260806 06:33:31.373522 27303 leader_election.cc:288] T 00000000000000000000000000000000 P 79d28096192a49218482b8f7ae5b25d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0cd57d823dae4f1e8a954ac15d7a76fc (127.7.43.125:44429), 3595b6a0d71242bb8b4a9f4781eb361e (127.7.43.124:46439)
I20260806 06:33:31.373757 27226 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "79d28096192a49218482b8f7ae5b25d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3595b6a0d71242bb8b4a9f4781eb361e" is_pre_election: true
I20260806 06:33:31.373790 27113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "79d28096192a49218482b8f7ae5b25d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cd57d823dae4f1e8a954ac15d7a76fc" is_pre_election: true
I20260806 06:33:31.373888 27226 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3595b6a0d71242bb8b4a9f4781eb361e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79d28096192a49218482b8f7ae5b25d9 in term 0.
I20260806 06:33:31.373930 27113 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0cd57d823dae4f1e8a954ac15d7a76fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79d28096192a49218482b8f7ae5b25d9 in term 0.
I20260806 06:33:31.374076 26977 leader_election.cc:302] T 00000000000000000000000000000000 P 79d28096192a49218482b8f7ae5b25d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3595b6a0d71242bb8b4a9f4781eb361e, 79d28096192a49218482b8f7ae5b25d9; no voters:
I20260806 06:33:31.374238 27303 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79d28096192a49218482b8f7ae5b25d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:33:31.374297 27303 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79d28096192a49218482b8f7ae5b25d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:33:31.374313 27303 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79d28096192a49218482b8f7ae5b25d9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:33:31.375003 27303 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79d28096192a49218482b8f7ae5b25d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79d28096192a49218482b8f7ae5b25d9" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 34003 } } peers { permanent_uuid: "0cd57d823dae4f1e8a954ac15d7a76fc" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 44429 } } peers { permanent_uuid: "3595b6a0d71242bb8b4a9f4781eb361e" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 46439 } }
I20260806 06:33:31.375118 27303 leader_election.cc:288] T 00000000000000000000000000000000 P 79d28096192a49218482b8f7ae5b25d9 [CANDIDATE]: Term 1 election: Requested vote from peers 0cd57d823dae4f1e8a954ac15d7a76fc (127.7.43.125:44429), 3595b6a0d71242bb8b4a9f4781eb361e (127.7.43.124:46439)
I20260806 06:33:31.375237 27113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "79d28096192a49218482b8f7ae5b25d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cd57d823dae4f1e8a954ac15d7a76fc"
I20260806 06:33:31.375289 27226 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "79d28096192a49218482b8f7ae5b25d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3595b6a0d71242bb8b4a9f4781eb361e"
I20260806 06:33:31.375317 27113 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0cd57d823dae4f1e8a954ac15d7a76fc [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:33:31.375352 27226 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3595b6a0d71242bb8b4a9f4781eb361e [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:33:31.375991 27113 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0cd57d823dae4f1e8a954ac15d7a76fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79d28096192a49218482b8f7ae5b25d9 in term 1.
I20260806 06:33:31.376003 27226 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3595b6a0d71242bb8b4a9f4781eb361e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79d28096192a49218482b8f7ae5b25d9 in term 1.
I20260806 06:33:31.376153 26981 leader_election.cc:302] T 00000000000000000000000000000000 P 79d28096192a49218482b8f7ae5b25d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cd57d823dae4f1e8a954ac15d7a76fc, 79d28096192a49218482b8f7ae5b25d9; no voters:
I20260806 06:33:31.376241 27303 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79d28096192a49218482b8f7ae5b25d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:33:31.376345 27303 raft_consensus.cc:686] T 00000000000000000000000000000000 P 79d28096192a49218482b8f7ae5b25d9 [term 1 LEADER]: Becoming Leader. State: Replica: 79d28096192a49218482b8f7ae5b25d9, State: Running, Role: LEADER
I20260806 06:33:31.376420 27303 consensus_queue.cc:237] T 00000000000000000000000000000000 P 79d28096192a49218482b8f7ae5b25d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79d28096192a49218482b8f7ae5b25d9" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 34003 } } peers { permanent_uuid: "0cd57d823dae4f1e8a954ac15d7a76fc" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 44429 } } peers { permanent_uuid: "3595b6a0d71242bb8b4a9f4781eb361e" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 46439 } }
I20260806 06:33:31.376765 27347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79d28096192a49218482b8f7ae5b25d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79d28096192a49218482b8f7ae5b25d9. Latest consensus state: current_term: 1 leader_uuid: "79d28096192a49218482b8f7ae5b25d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79d28096192a49218482b8f7ae5b25d9" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 34003 } } peers { permanent_uuid: "0cd57d823dae4f1e8a954ac15d7a76fc" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 44429 } } peers { permanent_uuid: "3595b6a0d71242bb8b4a9f4781eb361e" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 46439 } } }
I20260806 06:33:31.376856 27347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79d28096192a49218482b8f7ae5b25d9 [sys.catalog]: This master's current role is: LEADER
I20260806 06:33:31.377043 27349 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:33:31.377210 27349 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:33:31.378485 27226 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3595b6a0d71242bb8b4a9f4781eb361e [term 1 FOLLOWER]: Refusing update from remote peer 79d28096192a49218482b8f7ae5b25d9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:33:31.378724 27113 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0cd57d823dae4f1e8a954ac15d7a76fc [term 1 FOLLOWER]: Refusing update from remote peer 79d28096192a49218482b8f7ae5b25d9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:33:31.378798 27347 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 79d28096192a49218482b8f7ae5b25d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3595b6a0d71242bb8b4a9f4781eb361e" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 46439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:33:31.378959 27303 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 79d28096192a49218482b8f7ae5b25d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cd57d823dae4f1e8a954ac15d7a76fc" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 44429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:33:31.379994 27283 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3595b6a0d71242bb8b4a9f4781eb361e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79d28096192a49218482b8f7ae5b25d9. Latest consensus state: current_term: 1 leader_uuid: "79d28096192a49218482b8f7ae5b25d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79d28096192a49218482b8f7ae5b25d9" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 34003 } } peers { permanent_uuid: "0cd57d823dae4f1e8a954ac15d7a76fc" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 44429 } } peers { permanent_uuid: "3595b6a0d71242bb8b4a9f4781eb361e" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 46439 } } }
I20260806 06:33:31.379983 27330 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cd57d823dae4f1e8a954ac15d7a76fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79d28096192a49218482b8f7ae5b25d9. Latest consensus state: current_term: 1 leader_uuid: "79d28096192a49218482b8f7ae5b25d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79d28096192a49218482b8f7ae5b25d9" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 34003 } } peers { permanent_uuid: "0cd57d823dae4f1e8a954ac15d7a76fc" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 44429 } } peers { permanent_uuid: "3595b6a0d71242bb8b4a9f4781eb361e" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 46439 } } }
I20260806 06:33:31.380070 27330 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cd57d823dae4f1e8a954ac15d7a76fc [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:33:31.380070 27283 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3595b6a0d71242bb8b4a9f4781eb361e [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:33:31.380944 27347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79d28096192a49218482b8f7ae5b25d9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "79d28096192a49218482b8f7ae5b25d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79d28096192a49218482b8f7ae5b25d9" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 34003 } } peers { permanent_uuid: "0cd57d823dae4f1e8a954ac15d7a76fc" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 44429 } } peers { permanent_uuid: "3595b6a0d71242bb8b4a9f4781eb361e" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 46439 } } }
I20260806 06:33:31.380973 27303 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79d28096192a49218482b8f7ae5b25d9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "79d28096192a49218482b8f7ae5b25d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79d28096192a49218482b8f7ae5b25d9" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 34003 } } peers { permanent_uuid: "0cd57d823dae4f1e8a954ac15d7a76fc" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 44429 } } peers { permanent_uuid: "3595b6a0d71242bb8b4a9f4781eb361e" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 46439 } } }
I20260806 06:33:31.381033 27347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79d28096192a49218482b8f7ae5b25d9 [sys.catalog]: This master's current role is: LEADER
I20260806 06:33:31.381042 27303 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79d28096192a49218482b8f7ae5b25d9 [sys.catalog]: This master's current role is: LEADER
I20260806 06:33:31.381072 27330 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cd57d823dae4f1e8a954ac15d7a76fc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "79d28096192a49218482b8f7ae5b25d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79d28096192a49218482b8f7ae5b25d9" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 34003 } } peers { permanent_uuid: "0cd57d823dae4f1e8a954ac15d7a76fc" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 44429 } } peers { permanent_uuid: "3595b6a0d71242bb8b4a9f4781eb361e" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 46439 } } }
I20260806 06:33:31.381089 27351 mvcc.cc:205] Tried to move back new op lower bound from 7315447854604120064 to 7315447854598340608. Current Snapshot: MvccSnapshot[applied={T|T < 7315447854604120064}]
I20260806 06:33:31.381160 27330 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cd57d823dae4f1e8a954ac15d7a76fc [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:33:31.381456 27283 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3595b6a0d71242bb8b4a9f4781eb361e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "79d28096192a49218482b8f7ae5b25d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79d28096192a49218482b8f7ae5b25d9" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 34003 } } peers { permanent_uuid: "0cd57d823dae4f1e8a954ac15d7a76fc" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 44429 } } peers { permanent_uuid: "3595b6a0d71242bb8b4a9f4781eb361e" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 46439 } } }
I20260806 06:33:31.381536 27283 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3595b6a0d71242bb8b4a9f4781eb361e [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:33:31.381937 27349 catalog_manager.cc:1347] Generated new cluster ID: 6d4a6b8820ee4237bad9ffe6c83220d6
I20260806 06:33:31.381978 27349 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:33:31.396080 27349 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:33:31.396667 27349 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:33:31.401605 27349 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 79d28096192a49218482b8f7ae5b25d9: Generated new TSK 0
I20260806 06:33:31.401826 27349 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:33:32.006523 27298 catalog_manager.cc:1259] Loaded cluster ID: 6d4a6b8820ee4237bad9ffe6c83220d6
I20260806 06:33:32.006583 27298 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3595b6a0d71242bb8b4a9f4781eb361e: loading cluster ID for follower catalog manager: success
I20260806 06:33:32.007323 27298 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3595b6a0d71242bb8b4a9f4781eb361e: acquiring CA information for follower catalog manager: success
I20260806 06:33:32.007567 27298 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3595b6a0d71242bb8b4a9f4781eb361e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 06:33:32.068353 27343 catalog_manager.cc:1259] Loaded cluster ID: 6d4a6b8820ee4237bad9ffe6c83220d6
I20260806 06:33:32.068411 27343 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0cd57d823dae4f1e8a954ac15d7a76fc: loading cluster ID for follower catalog manager: success
I20260806 06:33:32.069169 27343 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0cd57d823dae4f1e8a954ac15d7a76fc: acquiring CA information for follower catalog manager: success
I20260806 06:33:32.069422 27343 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0cd57d823dae4f1e8a954ac15d7a76fc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 06:33:32.460157 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 26953
I20260806 06:33:32.472874 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 27058
I20260806 06:33:32.480302 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 27163
2026-08-06T06:33:32Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3148 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785997861828271-7341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:33:32 dist-test-slave-6zbq krb5kdc[27369](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:33:32 dist-test-slave-6zbq krb5kdc[27369](info): set up 2 sockets
Aug 06 06:33:32 dist-test-slave-6zbq krb5kdc[27369](info): commencing operation
krb5kdc: starting...
W20260806 06:33:33.588068 7341 mini_kdc.cc:121] Time spent starting KDC: real 1.086s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Aug 06 06:33:33 dist-test-slave-6zbq krb5kdc[27369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998013, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:33:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:33:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.43.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
I20260806 06:33:33.692610 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:36665
--webserver_interface=127.7.43.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:40583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.43.126:36665
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:33:33.770093 27389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:33:33.770270 27389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:33:33.770290 27389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:33:33.771386 27389 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:33:33.771425 27389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:33:33.771438 27389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:33:33.771449 27389 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:33:33.771461 27389 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:33:33.772831 27389 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:40583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785997861828271-7341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.43.126:36665
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:36665
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.43.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:33:33.773002 27389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:33:33.773165 27389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:33:33.777511 27403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:33:33.777673 27389 server_base.cc:1034] running on GCE node
W20260806 06:33:33.777478 27396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:33:33.777518 27397 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:33:33.777974 27389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:33:33.778240 27389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:33:33.779376 27389 hybrid_clock.cc:648] HybridClock initialized: now 1785998013779361 us; error 31 us; skew 500 ppm
Aug 06 06:33:33 dist-test-slave-6zbq krb5kdc[27369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998013, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:33:33.781555 27389 init.cc:377] Logged in from keytab as kudu/127.7.43.126@KRBTEST.COM (short username kudu)
I20260806 06:33:33.781926 27389 webserver.cc:460] Webserver started at http://127.7.43.126:41803/ using document root <none> and password file <none>
I20260806 06:33:33.782124 27389 fs_manager.cc:359] Metadata directory not provided
I20260806 06:33:33.782172 27389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:33:33.782291 27389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:33:33.783223 27389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785997861828271-7341-0/minicluster-data/master-0/data/instance:
uuid: "908099cf93464f268e2166c0653cbe78"
format_stamp: "Formatted at 2026-08-06 06:33:33 on dist-test-slave-6zbq"
server_key: "fd1b81a1eb3bc5582f8da38f73702388"
server_key_iv: "5d88601d4618e1375b439315d8850694"
server_key_version: "encryptionkey@0"
I20260806 06:33:33.783517 27389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785997861828271-7341-0/minicluster-data/master-0/wal/instance:
uuid: "908099cf93464f268e2166c0653cbe78"
format_stamp: "Formatted at 2026-08-06 06:33:33 on dist-test-slave-6zbq"
server_key: "fd1b81a1eb3bc5582f8da38f73702388"
server_key_iv: "5d88601d4618e1375b439315d8850694"
server_key_version: "encryptionkey@0"
I20260806 06:33:33.784643 27389 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:33:33.785239 27415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:33.785377 27389 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:33:33.785456 27389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "908099cf93464f268e2166c0653cbe78"
format_stamp: "Formatted at 2026-08-06 06:33:33 on dist-test-slave-6zbq"
server_key: "fd1b81a1eb3bc5582f8da38f73702388"
server_key_iv: "5d88601d4618e1375b439315d8850694"
server_key_version: "encryptionkey@0"
I20260806 06:33:33.785513 27389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:33:33.813772 27389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:33:33.815459 27389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:33:33.815570 27389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:33:33.819195 27389 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.126:36665
I20260806 06:33:33.819248 27493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.126:36665 every 8 connection(s)
I20260806 06:33:33.819523 27389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:33:33.820295 27494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:33:33.822398 27494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 908099cf93464f268e2166c0653cbe78: Bootstrap starting.
I20260806 06:33:33.822901 27494 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 908099cf93464f268e2166c0653cbe78: Neither blocks nor log segments found. Creating new log.
I20260806 06:33:33.823132 27494 log.cc:824] T 00000000000000000000000000000000 P 908099cf93464f268e2166c0653cbe78: Log is configured to *not* fsync() on all Append() calls
I20260806 06:33:33.823736 27494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 908099cf93464f268e2166c0653cbe78: No bootstrap required, opened a new log
I20260806 06:33:33.825196 27494 raft_consensus.cc:348] T 00000000000000000000000000000000 P 908099cf93464f268e2166c0653cbe78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "908099cf93464f268e2166c0653cbe78" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36665 } }
I20260806 06:33:33.825322 27494 raft_consensus.cc:374] T 00000000000000000000000000000000 P 908099cf93464f268e2166c0653cbe78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:33:33.825342 27494 raft_consensus.cc:729] T 00000000000000000000000000000000 P 908099cf93464f268e2166c0653cbe78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 908099cf93464f268e2166c0653cbe78, State: Initialized, Role: FOLLOWER
I20260806 06:33:33.825445 27494 consensus_queue.cc:260] T 00000000000000000000000000000000 P 908099cf93464f268e2166c0653cbe78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "908099cf93464f268e2166c0653cbe78" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36665 } }
I20260806 06:33:33.825515 27494 raft_consensus.cc:388] T 00000000000000000000000000000000 P 908099cf93464f268e2166c0653cbe78 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:33:33.825543 27494 raft_consensus.cc:482] T 00000000000000000000000000000000 P 908099cf93464f268e2166c0653cbe78 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:33:33.825573 27494 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 908099cf93464f268e2166c0653cbe78 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:33:33.826020 27494 raft_consensus.cc:504] T 00000000000000000000000000000000 P 908099cf93464f268e2166c0653cbe78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "908099cf93464f268e2166c0653cbe78" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36665 } }
I20260806 06:33:33.826098 27494 leader_election.cc:302] T 00000000000000000000000000000000 P 908099cf93464f268e2166c0653cbe78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 908099cf93464f268e2166c0653cbe78; no voters:
I20260806 06:33:33.826217 27494 leader_election.cc:288] T 00000000000000000000000000000000 P 908099cf93464f268e2166c0653cbe78 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:33:33.826260 27498 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 908099cf93464f268e2166c0653cbe78 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:33:33.826452 27498 raft_consensus.cc:686] T 00000000000000000000000000000000 P 908099cf93464f268e2166c0653cbe78 [term 1 LEADER]: Becoming Leader. State: Replica: 908099cf93464f268e2166c0653cbe78, State: Running, Role: LEADER
I20260806 06:33:33.826468 27494 sys_catalog.cc:564] T 00000000000000000000000000000000 P 908099cf93464f268e2166c0653cbe78 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:33:33.826541 27498 consensus_queue.cc:237] T 00000000000000000000000000000000 P 908099cf93464f268e2166c0653cbe78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "908099cf93464f268e2166c0653cbe78" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36665 } }
I20260806 06:33:33.826704 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 27389
I20260806 06:33:33.826808 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785997861828271-7341-0/minicluster-data/master-0/wal/instance
I20260806 06:33:33.827011 7341 external_mini_cluster.cc:1428] Setting key d731ab8bc111ef7205a789a5595a09a2
I20260806 06:33:33.827266 27499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 908099cf93464f268e2166c0653cbe78 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "908099cf93464f268e2166c0653cbe78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "908099cf93464f268e2166c0653cbe78" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36665 } } }
I20260806 06:33:33.827354 27499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 908099cf93464f268e2166c0653cbe78 [sys.catalog]: This master's current role is: LEADER
I20260806 06:33:33.827509 27501 sys_catalog.cc:455] T 00000000000000000000000000000000 P 908099cf93464f268e2166c0653cbe78 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 908099cf93464f268e2166c0653cbe78. Latest consensus state: current_term: 1 leader_uuid: "908099cf93464f268e2166c0653cbe78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "908099cf93464f268e2166c0653cbe78" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36665 } } }
I20260806 06:33:33.827608 27501 sys_catalog.cc:458] T 00000000000000000000000000000000 P 908099cf93464f268e2166c0653cbe78 [sys.catalog]: This master's current role is: LEADER
I20260806 06:33:33.827718 27513 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:33:33.828171 27513 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:33:33.829639 27513 catalog_manager.cc:1347] Generated new cluster ID: 346243c4dcf84146b5d747db46d5338b
I20260806 06:33:33.829699 27513 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:33:33 dist-test-slave-6zbq krb5kdc[27369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998013, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:33:33.838155 27522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:33.827725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38658 (local address 127.7.43.126:36665)
0806 06:33:33.827937 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:33.827991 (+ 54us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:33.828023 (+ 32us) server_negotiation.cc:407] Connection header received
0806 06:33:33.828197 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:33.828205 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:33.828269 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:33.828401 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:33:33.829181 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:33.830495 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:33.831546 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:33.832113 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:33.832216 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:33.834784 (+ 2568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:33:33.834788 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:33:33.834788 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:33:33.834800 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:33:33.836625 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:33.836979 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:33.836982 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:33.836983 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:33.837100 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:33.837282 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:33.837283 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:33.837283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:33.837574 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:33:33.837724 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:33.837804 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 06:33:33.837935 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":70,"threads_started":1}
I20260806 06:33:33.839262 7341 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:33:33.841053 27513 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:33:33.841746 27513 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:33:33.848153 27513 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 908099cf93464f268e2166c0653cbe78: Generated new TSK 0
I20260806 06:33:33.848310 27513 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:33:33.893129 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 27389
2026-08-06T06:33:33Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1417 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:33:33 dist-test-slave-6zbq krb5kdc[27531](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:33:33 dist-test-slave-6zbq krb5kdc[27531](info): set up 2 sockets
Aug 06 06:33:33 dist-test-slave-6zbq krb5kdc[27531](info): commencing operation
krb5kdc: starting...
W20260806 06:33:35.004549 7341 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Aug 06 06:33:35 dist-test-slave-6zbq krb5kdc[27531](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:33:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:33:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.43.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
I20260806 06:33:35.110045 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:40255
--webserver_interface=127.7.43.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:35907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.43.126:40255
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:33:35.188056 27551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:33:35.188243 27551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:33:35.188273 27551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:33:35.189354 27551 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:33:35.189393 27551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:33:35.189412 27551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:33:35.189427 27551 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:33:35.189445 27551 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:33:35.190862 27551 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:35907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.43.126:40255
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:40255
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.43.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27551
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:33:35.191083 27551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:33:35.191277 27551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:33:35.195640 27564 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:33:35.195698 27558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:33:35.195715 27551 server_base.cc:1034] running on GCE node
W20260806 06:33:35.195642 27559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:33:35.196033 27551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:33:35.196301 27551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:33:35.197435 27551 hybrid_clock.cc:648] HybridClock initialized: now 1785998015197419 us; error 33 us; skew 500 ppm
Aug 06 06:33:35 dist-test-slave-6zbq krb5kdc[27531](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998015, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:33:35.199672 27551 init.cc:377] Logged in from keytab as kudu/127.7.43.126@KRBTEST.COM (short username kudu)
I20260806 06:33:35.200037 27551 webserver.cc:460] Webserver started at http://127.7.43.126:43163/ using document root <none> and password file <none>
I20260806 06:33:35.200237 27551 fs_manager.cc:359] Metadata directory not provided
I20260806 06:33:35.200282 27551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:33:35.200402 27551 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:33:35.201344 27551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/master-0/data/instance:
uuid: "6369a1d6b9c64eae826178c1635ec0af"
format_stamp: "Formatted at 2026-08-06 06:33:35 on dist-test-slave-6zbq"
server_key: "d8abb7236e4219fcec9e5c1ce7eb9c77"
server_key_iv: "8008cf321fad7bcd62423025b6735156"
server_key_version: "encryptionkey@0"
I20260806 06:33:35.201664 27551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/master-0/wal/instance:
uuid: "6369a1d6b9c64eae826178c1635ec0af"
format_stamp: "Formatted at 2026-08-06 06:33:35 on dist-test-slave-6zbq"
server_key: "d8abb7236e4219fcec9e5c1ce7eb9c77"
server_key_iv: "8008cf321fad7bcd62423025b6735156"
server_key_version: "encryptionkey@0"
I20260806 06:33:35.202885 27551 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:33:35.203562 27572 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:35.203727 27551 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:33:35.203799 27551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "6369a1d6b9c64eae826178c1635ec0af"
format_stamp: "Formatted at 2026-08-06 06:33:35 on dist-test-slave-6zbq"
server_key: "d8abb7236e4219fcec9e5c1ce7eb9c77"
server_key_iv: "8008cf321fad7bcd62423025b6735156"
server_key_version: "encryptionkey@0"
I20260806 06:33:35.203856 27551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:33:35.214499 27551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:33:35.216081 27551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:33:35.216200 27551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:33:35.219772 27551 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.126:40255
I20260806 06:33:35.219803 27650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.126:40255 every 8 connection(s)
I20260806 06:33:35.220073 27551 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:33:35.220906 27651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:33:35.222940 27651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af: Bootstrap starting.
I20260806 06:33:35.223428 27651 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af: Neither blocks nor log segments found. Creating new log.
I20260806 06:33:35.223642 27651 log.cc:824] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af: Log is configured to *not* fsync() on all Append() calls
I20260806 06:33:35.223893 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 27551
I20260806 06:33:35.223999 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/master-0/wal/instance
I20260806 06:33:35.224192 7341 external_mini_cluster.cc:1428] Setting key f2819d09446833d6c6b47636cdc1b65d
I20260806 06:33:35.224375 27651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af: No bootstrap required, opened a new log
I20260806 06:33:35.226101 27651 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6369a1d6b9c64eae826178c1635ec0af" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 40255 } }
I20260806 06:33:35.226241 27651 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:33:35.226274 27651 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6369a1d6b9c64eae826178c1635ec0af, State: Initialized, Role: FOLLOWER
I20260806 06:33:35.226410 27651 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6369a1d6b9c64eae826178c1635ec0af" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 40255 } }
I20260806 06:33:35.226471 27651 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:33:35.226500 27651 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:33:35.226545 27651 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:33:35.227018 27651 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6369a1d6b9c64eae826178c1635ec0af" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 40255 } }
I20260806 06:33:35.227118 27651 leader_election.cc:302] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6369a1d6b9c64eae826178c1635ec0af; no voters:
I20260806 06:33:35.227259 27651 leader_election.cc:288] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:33:35.227291 27658 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:33:35.227401 27658 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af [term 1 LEADER]: Becoming Leader. State: Replica: 6369a1d6b9c64eae826178c1635ec0af, State: Running, Role: LEADER
I20260806 06:33:35.227505 27651 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:33:35.227510 27658 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6369a1d6b9c64eae826178c1635ec0af" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 40255 } }
I20260806 06:33:35.227941 27658 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6369a1d6b9c64eae826178c1635ec0af. Latest consensus state: current_term: 1 leader_uuid: "6369a1d6b9c64eae826178c1635ec0af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6369a1d6b9c64eae826178c1635ec0af" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 40255 } } }
I20260806 06:33:35.228004 27660 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6369a1d6b9c64eae826178c1635ec0af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6369a1d6b9c64eae826178c1635ec0af" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 40255 } } }
I20260806 06:33:35.228040 27658 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af [sys.catalog]: This master's current role is: LEADER
I20260806 06:33:35.228096 27660 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af [sys.catalog]: This master's current role is: LEADER
I20260806 06:33:35.228899 27668 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:33:35.229419 27668 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:33:35.230958 27668 catalog_manager.cc:1347] Generated new cluster ID: c117f87d46394f5fb83dc34a236af890
I20260806 06:33:35.231014 27668 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:33:35 dist-test-slave-6zbq krb5kdc[27531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:33:35.234257 27656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:35.224625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53312 (local address 127.7.43.126:40255)
0806 06:33:35.224810 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:35.224818 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:35.224846 (+ 28us) server_negotiation.cc:407] Connection header received
0806 06:33:35.225492 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:35.225498 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:35.225543 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:35.225657 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:33:35.226248 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:35.227558 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:35.228531 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:35.229105 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:35.229206 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:35.231498 (+ 2292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:33:35.231503 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:33:35.231503 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:33:35.231514 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:33:35.232925 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:35.233238 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:35.233242 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:35.233242 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:35.233317 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:35.233487 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:35.233488 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:35.233488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:35.233693 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:33:35.233840 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:35.233942 (+ 102us) server_negotiation.cc:299] Negotiation successful
0806 06:33:35.234067 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":94,"threads_started":1}
I20260806 06:33:35.238492 27668 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:33:35.238974 27668 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:33:35.241524 27668 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af: Generated new TSK 0
I20260806 06:33:35.241657 27668 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.43.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab.
Entry for principal kudu/127.7.43.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab.
I20260806 06:33:35.268369 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.65:0
--local_ip_for_outbound_sockets=127.7.43.65
--webserver_interface=127.7.43.65
--webserver_port=0
--tserver_master_addrs=127.7.43.126:40255
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab
--principal=kudu/127.7.43.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:35907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:33:35.368952 27687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:33:35.369132 27687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:33:35.369161 27687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:33:35.370250 27687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:33:35.370290 27687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:33:35.370325 27687 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.65
I20260806 06:33:35.371779 27687 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:35907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab
--principal=kudu/127.7.43.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.43.65
--webserver_port=0
--tserver_master_addrs=127.7.43.126:40255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27687
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.65
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:33:35.371995 27687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:33:35.372187 27687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:33:35.377521 27692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:33:35.377616 27693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:33:35.377507 27695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:33:35.377772 27687 server_base.cc:1034] running on GCE node
I20260806 06:33:35.377980 27687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:33:35.378232 27687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:33:35.379379 27687 hybrid_clock.cc:648] HybridClock initialized: now 1785998015379366 us; error 31 us; skew 500 ppm
Aug 06 06:33:35 dist-test-slave-6zbq krb5kdc[27531](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998015, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:33:35.381582 27687 init.cc:377] Logged in from keytab as kudu/127.7.43.65@KRBTEST.COM (short username kudu)
I20260806 06:33:35.381959 27687 webserver.cc:460] Webserver started at http://127.7.43.65:42865/ using document root <none> and password file <none>
I20260806 06:33:35.382159 27687 fs_manager.cc:359] Metadata directory not provided
I20260806 06:33:35.382205 27687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:33:35.382320 27687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:33:35.383249 27687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-0/data/instance:
uuid: "188fbd273bd64a288842883839b5326a"
format_stamp: "Formatted at 2026-08-06 06:33:35 on dist-test-slave-6zbq"
server_key: "92ec15286eb03bb790eb48a65176b51a"
server_key_iv: "54c111ae30fb539743e801216b9f9058"
server_key_version: "encryptionkey@0"
I20260806 06:33:35.383549 27687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-0/wal/instance:
uuid: "188fbd273bd64a288842883839b5326a"
format_stamp: "Formatted at 2026-08-06 06:33:35 on dist-test-slave-6zbq"
server_key: "92ec15286eb03bb790eb48a65176b51a"
server_key_iv: "54c111ae30fb539743e801216b9f9058"
server_key_version: "encryptionkey@0"
I20260806 06:33:35.384677 27687 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:33:35.385294 27705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:35.385439 27687 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:33:35.385519 27687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-0/wal
uuid: "188fbd273bd64a288842883839b5326a"
format_stamp: "Formatted at 2026-08-06 06:33:35 on dist-test-slave-6zbq"
server_key: "92ec15286eb03bb790eb48a65176b51a"
server_key_iv: "54c111ae30fb539743e801216b9f9058"
server_key_version: "encryptionkey@0"
I20260806 06:33:35.385576 27687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:33:35.392930 27687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:33:35.394551 27687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:33:35.394670 27687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:33:35.394855 27687 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:33:35.395113 27687 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:33:35.395146 27687 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:35.395171 27687 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:33:35.395187 27687 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:35.400434 27687 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.65:42385
I20260806 06:33:35.400493 27885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.65:42385 every 8 connection(s)
I20260806 06:33:35.400746 27687 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
I20260806 06:33:35.402897 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 27687
I20260806 06:33:35.403004 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-0/wal/instance
I20260806 06:33:35.403196 7341 external_mini_cluster.cc:1428] Setting key b8c63f02449a119dbac1628c7b5c9f30
Aug 06 06:33:35 dist-test-slave-6zbq krb5kdc[27531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998015, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.65@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:33:35.409502 27656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:35.401320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:55987 (local address 127.7.43.126:40255)
0806 06:33:35.401361 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:35.401363 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:35.401519 (+ 156us) server_negotiation.cc:407] Connection header received
0806 06:33:35.401960 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:35.401961 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:35.401970 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:35.402008 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:33:35.402893 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:35.403545 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:35.404523 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:35.404795 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:35.404864 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:35.406993 (+ 2129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:33:35.406994 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:33:35.406995 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:33:35.407002 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:33:35.408237 (+ 1235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:35.408662 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:35.408662 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:35.408663 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:35.408715 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:35.408961 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:35.408962 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:35.408962 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:35.409050 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:33:35.409089 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:35.409384 (+ 295us) server_negotiation.cc:299] Negotiation successful
0806 06:33:35.409441 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:33:35.409752 27888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:35.401272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.65:55987)
0806 06:33:35.401417 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:35.401453 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 06:33:35.401876 (+ 423us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:33:35.402069 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:33:35.402072 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:33:35.402640 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:33:35.402808 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:35.402813 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:35.403633 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:35.403634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:35.404442 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:35.404445 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:35.404865 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:35.404866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:35.404958 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:35.405380 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:33:35.405391 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:33:35.406908 (+ 1517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:33:35.408299 (+ 1391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:33:35.408307 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:33:35.408309 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:33:35.408592 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:33:35.408771 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:33:35.408771 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:33:35.408772 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:33:35.408902 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:33:35.409102 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:33:35.409108 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:33:35.409260 (+ 152us) client_negotiation.cc:769] Sending connection context
0806 06:33:35.409375 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 06:33:35.409531 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":77,"threads_started":1}
I20260806 06:33:35.410192 27886 heartbeater.cc:347] Connected to a master server at 127.7.43.126:40255
I20260806 06:33:35.410286 27886 heartbeater.cc:464] Registering TS with master...
I20260806 06:33:35.410498 27886 heartbeater.cc:511] Master 127.7.43.126:40255 requested a full tablet report, sending...
I20260806 06:33:35.410957 27591 ts_manager.cc:194] Registered new tserver with Master: 188fbd273bd64a288842883839b5326a (127.7.43.65:42385)
I20260806 06:33:35.411518 27591 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.65@KRBTEST.COM'} at 127.7.43.65:55987
WARNING: no policy specified for kudu/127.7.43.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab.
Entry for principal kudu/127.7.43.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab.
I20260806 06:33:35.426206 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.66:0
--local_ip_for_outbound_sockets=127.7.43.66
--webserver_interface=127.7.43.66
--webserver_port=0
--tserver_master_addrs=127.7.43.126:40255
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab
--principal=kudu/127.7.43.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:35907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:33:35.505345 27893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:33:35.505523 27893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:33:35.505544 27893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:33:35.506685 27893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:33:35.506722 27893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:33:35.506748 27893 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.66
I20260806 06:33:35.508164 27893 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:35907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab
--principal=kudu/127.7.43.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.43.66
--webserver_port=0
--tserver_master_addrs=127.7.43.126:40255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27893
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.66
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:33:35.508327 27893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:33:35.508479 27893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:33:35.513032 27900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:33:35.513033 27905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:33:35.513032 27902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:33:35.513226 27893 server_base.cc:1034] running on GCE node
I20260806 06:33:35.513402 27893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:33:35.513610 27893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:33:35.514739 27893 hybrid_clock.cc:648] HybridClock initialized: now 1785998015514723 us; error 29 us; skew 500 ppm
Aug 06 06:33:35 dist-test-slave-6zbq krb5kdc[27531](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998015, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:33:35.516966 27893 init.cc:377] Logged in from keytab as kudu/127.7.43.66@KRBTEST.COM (short username kudu)
I20260806 06:33:35.517349 27893 webserver.cc:460] Webserver started at http://127.7.43.66:45099/ using document root <none> and password file <none>
I20260806 06:33:35.517529 27893 fs_manager.cc:359] Metadata directory not provided
I20260806 06:33:35.517575 27893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:33:35.517691 27893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:33:35.518683 27893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-1/data/instance:
uuid: "02ad740aae2d470cafcb96d12f589902"
format_stamp: "Formatted at 2026-08-06 06:33:35 on dist-test-slave-6zbq"
server_key: "d3c084445a41564da8f73c40ecf31c75"
server_key_iv: "a3eb948d1b8ee2f36fe8d497a026ea36"
server_key_version: "encryptionkey@0"
I20260806 06:33:35.518996 27893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-1/wal/instance:
uuid: "02ad740aae2d470cafcb96d12f589902"
format_stamp: "Formatted at 2026-08-06 06:33:35 on dist-test-slave-6zbq"
server_key: "d3c084445a41564da8f73c40ecf31c75"
server_key_iv: "a3eb948d1b8ee2f36fe8d497a026ea36"
server_key_version: "encryptionkey@0"
I20260806 06:33:35.520128 27893 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:33:35.520766 27914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:35.520920 27893 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:33:35.521008 27893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-1/wal
uuid: "02ad740aae2d470cafcb96d12f589902"
format_stamp: "Formatted at 2026-08-06 06:33:35 on dist-test-slave-6zbq"
server_key: "d3c084445a41564da8f73c40ecf31c75"
server_key_iv: "a3eb948d1b8ee2f36fe8d497a026ea36"
server_key_version: "encryptionkey@0"
I20260806 06:33:35.521068 27893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:33:35.529855 27893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:33:35.531750 27893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:33:35.531874 27893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:33:35.532051 27893 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:33:35.532318 27893 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:33:35.532351 27893 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:35.532375 27893 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:33:35.532402 27893 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:35.537724 27893 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.66:38259
I20260806 06:33:35.537781 28064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.66:38259 every 8 connection(s)
I20260806 06:33:35.538023 27893 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
I20260806 06:33:35.540598 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 27893
I20260806 06:33:35.540694 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-1/wal/instance
I20260806 06:33:35.540859 7341 external_mini_cluster.cc:1428] Setting key f9eaae6e706b7c6782dd166ac6d9365f
Aug 06 06:33:35 dist-test-slave-6zbq krb5kdc[27531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998015, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.66@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:33:35.546056 27656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:35.538646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:60573 (local address 127.7.43.126:40255)
0806 06:33:35.538696 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:35.538698 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:35.538855 (+ 157us) server_negotiation.cc:407] Connection header received
0806 06:33:35.539252 (+ 397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:35.539253 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:35.539262 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:35.539313 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:33:35.540083 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:35.540574 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:35.541611 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:35.541811 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:35.541867 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:35.543759 (+ 1892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:33:35.543760 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:33:35.543761 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:33:35.543768 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:33:35.544872 (+ 1104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:35.545221 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:35.545222 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:35.545223 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:35.545275 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:35.545482 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:35.545483 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:35.545484 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:35.545587 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:33:35.545641 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:35.545890 (+ 249us) server_negotiation.cc:299] Negotiation successful
0806 06:33:35.545969 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 06:33:35.546202 28073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:35.538607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.66:60573)
0806 06:33:35.538745 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:35.538783 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 06:33:35.539180 (+ 397us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:33:35.539360 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:33:35.539362 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:33:35.539855 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:33:35.540020 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:35.540024 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:35.540662 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:35.540664 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:35.541517 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:35.541520 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:35.541884 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:35.541885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:35.542007 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:35.542442 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:33:35.542449 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:33:35.543670 (+ 1221us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:33:35.544944 (+ 1274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:33:35.544951 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:33:35.544953 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:33:35.545154 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:33:35.545330 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:33:35.545331 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:33:35.545331 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:33:35.545425 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:33:35.545642 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:33:35.545648 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:33:35.545772 (+ 124us) client_negotiation.cc:769] Sending connection context
0806 06:33:35.545861 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 06:33:35.546006 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":74,"threads_started":1}
I20260806 06:33:35.546653 28065 heartbeater.cc:347] Connected to a master server at 127.7.43.126:40255
I20260806 06:33:35.546733 28065 heartbeater.cc:464] Registering TS with master...
I20260806 06:33:35.546890 28065 heartbeater.cc:511] Master 127.7.43.126:40255 requested a full tablet report, sending...
I20260806 06:33:35.547243 27591 ts_manager.cc:194] Registered new tserver with Master: 02ad740aae2d470cafcb96d12f589902 (127.7.43.66:38259)
I20260806 06:33:35.547729 27591 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.66@KRBTEST.COM'} at 127.7.43.66:60573
WARNING: no policy specified for kudu/127.7.43.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab.
Entry for principal kudu/127.7.43.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab.
I20260806 06:33:35.563225 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.67:0
--local_ip_for_outbound_sockets=127.7.43.67
--webserver_interface=127.7.43.67
--webserver_port=0
--tserver_master_addrs=127.7.43.126:40255
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab
--principal=kudu/127.7.43.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:35907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:33:35.641644 28076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:33:35.641826 28076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:33:35.641847 28076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:33:35.642926 28076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:33:35.642963 28076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:33:35.642987 28076 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.67
I20260806 06:33:35.644312 28076 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:35907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab
--principal=kudu/127.7.43.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.43.67
--webserver_port=0
--tserver_master_addrs=127.7.43.126:40255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28076
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.67
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:33:35.644474 28076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:33:35.644629 28076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:33:35.649103 28088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:33:35.649091 28086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:33:35.649121 28076 server_base.cc:1034] running on GCE node
W20260806 06:33:35.649091 28085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:33:35.649468 28076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:33:35.649679 28076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:33:35.650825 28076 hybrid_clock.cc:648] HybridClock initialized: now 1785998015650806 us; error 29 us; skew 500 ppm
Aug 06 06:33:35 dist-test-slave-6zbq krb5kdc[27531](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998015, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:33:35.657728 28076 init.cc:377] Logged in from keytab as kudu/127.7.43.67@KRBTEST.COM (short username kudu)
I20260806 06:33:35.658083 28076 webserver.cc:460] Webserver started at http://127.7.43.67:39839/ using document root <none> and password file <none>
I20260806 06:33:35.658245 28076 fs_manager.cc:359] Metadata directory not provided
I20260806 06:33:35.658284 28076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:33:35.658416 28076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:33:35.659340 28076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-2/data/instance:
uuid: "194dbcf584f04a608c7f6a6a251d71e5"
format_stamp: "Formatted at 2026-08-06 06:33:35 on dist-test-slave-6zbq"
server_key: "4d996aa994a70e5b70ed78697afbc27c"
server_key_iv: "0e2e1d01e6bd2bb93a2cedfca6252a31"
server_key_version: "encryptionkey@0"
I20260806 06:33:35.659614 28076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-2/wal/instance:
uuid: "194dbcf584f04a608c7f6a6a251d71e5"
format_stamp: "Formatted at 2026-08-06 06:33:35 on dist-test-slave-6zbq"
server_key: "4d996aa994a70e5b70ed78697afbc27c"
server_key_iv: "0e2e1d01e6bd2bb93a2cedfca6252a31"
server_key_version: "encryptionkey@0"
I20260806 06:33:35.660728 28076 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:33:35.661338 28098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:35.661489 28076 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:33:35.661568 28076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-2/wal
uuid: "194dbcf584f04a608c7f6a6a251d71e5"
format_stamp: "Formatted at 2026-08-06 06:33:35 on dist-test-slave-6zbq"
server_key: "4d996aa994a70e5b70ed78697afbc27c"
server_key_iv: "0e2e1d01e6bd2bb93a2cedfca6252a31"
server_key_version: "encryptionkey@0"
I20260806 06:33:35.661619 28076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:33:35.680465 28076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:33:35.682327 28076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:33:35.682463 28076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:33:35.682626 28076 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:33:35.682859 28076 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:33:35.682888 28076 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:35.682911 28076 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:33:35.682927 28076 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:35.688153 28076 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.67:45915
I20260806 06:33:35.688239 28260 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.67:45915 every 8 connection(s)
I20260806 06:33:35.688472 28076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
Aug 06 06:33:35 dist-test-slave-6zbq krb5kdc[27531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998015, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.67@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:33:35.696168 27656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:35.689131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:59065 (local address 127.7.43.126:40255)
0806 06:33:35.689192 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:35.689194 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:35.689349 (+ 155us) server_negotiation.cc:407] Connection header received
0806 06:33:35.689765 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:35.689766 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:35.689776 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:35.689824 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:33:35.690703 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:35.691181 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:35.692065 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:35.692242 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:35.692289 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:35.694027 (+ 1738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:33:35.694028 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:33:35.694028 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:33:35.694035 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:33:35.695200 (+ 1165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:35.695521 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:35.695522 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:35.695522 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:35.695567 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:35.695733 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:35.695733 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:35.695733 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:35.695822 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:33:35.695866 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:35.696054 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 06:33:35.696107 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 06:33:35.696327 28264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:35.689079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.67:59065)
0806 06:33:35.689236 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:35.689274 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 06:33:35.689692 (+ 418us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:33:35.689877 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:33:35.689880 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:33:35.690462 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:33:35.690637 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:35.690642 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:35.691256 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:35.691256 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:35.692003 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:35.692005 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:35.692295 (+ 290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:35.692296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:35.692386 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:35.692748 (+ 362us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:33:35.692755 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:33:35.693952 (+ 1197us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:33:35.695265 (+ 1313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:33:35.695271 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:33:35.695272 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:33:35.695467 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:33:35.695610 (+ 143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:33:35.695610 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:33:35.695610 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:33:35.695692 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:33:35.695865 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:33:35.695868 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:33:35.695966 (+ 98us) client_negotiation.cc:769] Sending connection context
0806 06:33:35.696038 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 06:33:35.696143 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":78,"threads_started":1}
I20260806 06:33:35.696728 28261 heartbeater.cc:347] Connected to a master server at 127.7.43.126:40255
I20260806 06:33:35.696810 28261 heartbeater.cc:464] Registering TS with master...
I20260806 06:33:35.696938 28261 heartbeater.cc:511] Master 127.7.43.126:40255 requested a full tablet report, sending...
I20260806 06:33:35.697269 27591 ts_manager.cc:194] Registered new tserver with Master: 194dbcf584f04a608c7f6a6a251d71e5 (127.7.43.67:45915)
I20260806 06:33:35.697516 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 28076
I20260806 06:33:35.697607 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-2/wal/instance
I20260806 06:33:35.697674 27591 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.67@KRBTEST.COM'} at 127.7.43.67:59065
I20260806 06:33:35.697775 7341 external_mini_cluster.cc:1428] Setting key 67b34083be8d24715ac7524350d1e856
I20260806 06:33:35.698590 7341 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:33:35.704768 27656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:35.699806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53322 (local address 127.7.43.126:40255)
0806 06:33:35.699858 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:35.699860 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:35.699955 (+ 95us) server_negotiation.cc:407] Connection header received
0806 06:33:35.700016 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:35.700017 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:35.700026 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:35.700071 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:33:35.700589 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:35.701034 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:35.701742 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:35.701917 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:35.701968 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:35.702978 (+ 1010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:33:35.702979 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:33:35.702980 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:33:35.702986 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:33:35.703944 (+ 958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:35.704233 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:35.704234 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:35.704234 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:35.704276 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:35.704402 (+ 126us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:35.704402 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:35.704402 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:35.704512 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:33:35.704581 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:35.704662 (+ 81us) server_negotiation.cc:299] Negotiation successful
0806 06:33:35.704714 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 06:33:35.706036 27591 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53322:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:33:35.707602 27591 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:33:35.713306 28274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:35.709452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.66:38259 (local address 127.0.0.1:46124)
0806 06:33:35.709555 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:35.709593 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 06:33:35.709658 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:33:35.710282 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:33:35.710283 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:33:35.710285 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:33:35.710320 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:35.710321 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:35.711583 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:35.711584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:35.712317 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:35.712319 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:35.713014 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:35.713015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:35.713108 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:35.713120 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 06:33:35.713180 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 06:33:35.713239 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78,"thread_start_us":55,"threads_started":1}
I20260806 06:33:35.713315 28280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:35.709705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46124 (local address 127.7.43.66:38259)
0806 06:33:35.710100 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:35.710104 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:35.710114 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:33:35.710149 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:35.710151 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:35.710165 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:35.710420 (+ 255us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:33:35.710492 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:35.711501 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:35.712408 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:35.712934 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:35.712998 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:35.713055 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:35.713201 (+ 146us) server_negotiation.cc:299] Negotiation successful
0806 06:33:35.713239 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":89,"threads_started":1}
I20260806 06:33:35.713624 28279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:35.709634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:42385 (local address 127.0.0.1:42786)
0806 06:33:35.709862 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:35.709872 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:33:35.709909 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:33:35.710178 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:33:35.710179 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:33:35.710181 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:33:35.710234 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:35.710237 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:35.711179 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:35.711180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:35.712431 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:35.712434 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:35.713402 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:35.713403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:35.713465 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:35.713469 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:33:35.713511 (+ 42us) client_negotiation.cc:240] Negotiation successful
0806 06:33:35.713550 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"mutex_wait_us":38,"thread_start_us":85,"threads_started":1}
I20260806 06:33:35.713727 28275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:35.709452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.67:45915 (local address 127.0.0.1:59570)
0806 06:33:35.709590 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:35.709599 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:33:35.709823 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:33:35.710135 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:33:35.710138 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:33:35.710140 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:33:35.710199 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:35.710200 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:35.711434 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:35.711435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:35.712486 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:35.712488 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:35.713537 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:35.713538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:35.713598 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:35.713601 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:33:35.713637 (+ 36us) client_negotiation.cc:240] Negotiation successful
0806 06:33:35.713669 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":2,"spinlock_wait_cycles":384,"thread_start_us":57,"threads_started":1}
I20260806 06:33:35.713923 28281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:35.709890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42786 (local address 127.7.43.65:42385)
0806 06:33:35.710051 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:35.710055 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:35.710062 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:33:35.710093 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:35.710094 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:35.710105 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:35.710188 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:33:35.710298 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:35.711092 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:35.712527 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:35.713248 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:35.713315 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:35.713366 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:35.713806 (+ 440us) server_negotiation.cc:299] Negotiation successful
0806 06:33:35.713849 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":84,"threads_started":1}
I20260806 06:33:35.713959 28276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:35.709544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59570 (local address 127.7.43.67:45915)
0806 06:33:35.709915 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:35.709919 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:35.709930 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:33:35.709964 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:35.709966 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:35.709980 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:35.710056 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:33:35.710271 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:35.711337 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:35.712563 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:35.713248 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:35.713315 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:35.713366 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:35.713860 (+ 494us) server_negotiation.cc:299] Negotiation successful
0806 06:33:35.713897 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":82,"threads_started":1}
I20260806 06:33:35.714154 27977 tablet_service.cc:1467] Processing CreateTablet for tablet 9dd97b6fe0904d62873a10b942fc37c1 (DEFAULT_TABLE table=test-table [id=116800572cf84dc084de49d8c24de0dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:33:35.714536 27977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9dd97b6fe0904d62873a10b942fc37c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:33:35.714576 28170 tablet_service.cc:1467] Processing CreateTablet for tablet 9dd97b6fe0904d62873a10b942fc37c1 (DEFAULT_TABLE table=test-table [id=116800572cf84dc084de49d8c24de0dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:33:35.714566 27787 tablet_service.cc:1467] Processing CreateTablet for tablet 9dd97b6fe0904d62873a10b942fc37c1 (DEFAULT_TABLE table=test-table [id=116800572cf84dc084de49d8c24de0dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:33:35.714871 27787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9dd97b6fe0904d62873a10b942fc37c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:33:35.714871 28170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9dd97b6fe0904d62873a10b942fc37c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:33:35.717060 28282 tablet_bootstrap.cc:492] T 9dd97b6fe0904d62873a10b942fc37c1 P 02ad740aae2d470cafcb96d12f589902: Bootstrap starting.
I20260806 06:33:35.717876 28284 tablet_bootstrap.cc:492] T 9dd97b6fe0904d62873a10b942fc37c1 P 188fbd273bd64a288842883839b5326a: Bootstrap starting.
I20260806 06:33:35.718127 28282 tablet_bootstrap.cc:654] T 9dd97b6fe0904d62873a10b942fc37c1 P 02ad740aae2d470cafcb96d12f589902: Neither blocks nor log segments found. Creating new log.
I20260806 06:33:35.718482 28285 tablet_bootstrap.cc:492] T 9dd97b6fe0904d62873a10b942fc37c1 P 194dbcf584f04a608c7f6a6a251d71e5: Bootstrap starting.
I20260806 06:33:35.718662 28284 tablet_bootstrap.cc:654] T 9dd97b6fe0904d62873a10b942fc37c1 P 188fbd273bd64a288842883839b5326a: Neither blocks nor log segments found. Creating new log.
I20260806 06:33:35.718703 28282 log.cc:824] T 9dd97b6fe0904d62873a10b942fc37c1 P 02ad740aae2d470cafcb96d12f589902: Log is configured to *not* fsync() on all Append() calls
I20260806 06:33:35.719084 28285 tablet_bootstrap.cc:654] T 9dd97b6fe0904d62873a10b942fc37c1 P 194dbcf584f04a608c7f6a6a251d71e5: Neither blocks nor log segments found. Creating new log.
I20260806 06:33:35.719159 28284 log.cc:824] T 9dd97b6fe0904d62873a10b942fc37c1 P 188fbd273bd64a288842883839b5326a: Log is configured to *not* fsync() on all Append() calls
I20260806 06:33:35.719496 28282 tablet_bootstrap.cc:492] T 9dd97b6fe0904d62873a10b942fc37c1 P 02ad740aae2d470cafcb96d12f589902: No bootstrap required, opened a new log
I20260806 06:33:35.719558 28282 ts_tablet_manager.cc:1397] T 9dd97b6fe0904d62873a10b942fc37c1 P 02ad740aae2d470cafcb96d12f589902: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260806 06:33:35.719704 28285 log.cc:824] T 9dd97b6fe0904d62873a10b942fc37c1 P 194dbcf584f04a608c7f6a6a251d71e5: Log is configured to *not* fsync() on all Append() calls
I20260806 06:33:35.719924 28284 tablet_bootstrap.cc:492] T 9dd97b6fe0904d62873a10b942fc37c1 P 188fbd273bd64a288842883839b5326a: No bootstrap required, opened a new log
I20260806 06:33:35.720049 28284 ts_tablet_manager.cc:1397] T 9dd97b6fe0904d62873a10b942fc37c1 P 188fbd273bd64a288842883839b5326a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 06:33:35.720331 28285 tablet_bootstrap.cc:492] T 9dd97b6fe0904d62873a10b942fc37c1 P 194dbcf584f04a608c7f6a6a251d71e5: No bootstrap required, opened a new log
I20260806 06:33:35.720388 28285 ts_tablet_manager.cc:1397] T 9dd97b6fe0904d62873a10b942fc37c1 P 194dbcf584f04a608c7f6a6a251d71e5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:33:35.721477 28282 raft_consensus.cc:348] T 9dd97b6fe0904d62873a10b942fc37c1 P 02ad740aae2d470cafcb96d12f589902 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "194dbcf584f04a608c7f6a6a251d71e5" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 45915 } } peers { permanent_uuid: "02ad740aae2d470cafcb96d12f589902" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 38259 } } peers { permanent_uuid: "188fbd273bd64a288842883839b5326a" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 42385 } }
I20260806 06:33:35.721657 28282 raft_consensus.cc:374] T 9dd97b6fe0904d62873a10b942fc37c1 P 02ad740aae2d470cafcb96d12f589902 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:33:35.721695 28282 raft_consensus.cc:729] T 9dd97b6fe0904d62873a10b942fc37c1 P 02ad740aae2d470cafcb96d12f589902 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02ad740aae2d470cafcb96d12f589902, State: Initialized, Role: FOLLOWER
I20260806 06:33:35.721870 28282 consensus_queue.cc:260] T 9dd97b6fe0904d62873a10b942fc37c1 P 02ad740aae2d470cafcb96d12f589902 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "194dbcf584f04a608c7f6a6a251d71e5" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 45915 } } peers { permanent_uuid: "02ad740aae2d470cafcb96d12f589902" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 38259 } } peers { permanent_uuid: "188fbd273bd64a288842883839b5326a" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 42385 } }
I20260806 06:33:35.721885 28285 raft_consensus.cc:348] T 9dd97b6fe0904d62873a10b942fc37c1 P 194dbcf584f04a608c7f6a6a251d71e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "194dbcf584f04a608c7f6a6a251d71e5" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 45915 } } peers { permanent_uuid: "02ad740aae2d470cafcb96d12f589902" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 38259 } } peers { permanent_uuid: "188fbd273bd64a288842883839b5326a" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 42385 } }
I20260806 06:33:35.722076 28285 raft_consensus.cc:374] T 9dd97b6fe0904d62873a10b942fc37c1 P 194dbcf584f04a608c7f6a6a251d71e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:33:35.722132 28285 raft_consensus.cc:729] T 9dd97b6fe0904d62873a10b942fc37c1 P 194dbcf584f04a608c7f6a6a251d71e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 194dbcf584f04a608c7f6a6a251d71e5, State: Initialized, Role: FOLLOWER
I20260806 06:33:35.722024 28284 raft_consensus.cc:348] T 9dd97b6fe0904d62873a10b942fc37c1 P 188fbd273bd64a288842883839b5326a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "194dbcf584f04a608c7f6a6a251d71e5" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 45915 } } peers { permanent_uuid: "02ad740aae2d470cafcb96d12f589902" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 38259 } } peers { permanent_uuid: "188fbd273bd64a288842883839b5326a" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 42385 } }
I20260806 06:33:35.722153 28282 ts_tablet_manager.cc:1428] T 9dd97b6fe0904d62873a10b942fc37c1 P 02ad740aae2d470cafcb96d12f589902: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260806 06:33:35.722182 28284 raft_consensus.cc:374] T 9dd97b6fe0904d62873a10b942fc37c1 P 188fbd273bd64a288842883839b5326a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:33:35.722234 28284 raft_consensus.cc:729] T 9dd97b6fe0904d62873a10b942fc37c1 P 188fbd273bd64a288842883839b5326a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 188fbd273bd64a288842883839b5326a, State: Initialized, Role: FOLLOWER
I20260806 06:33:35.722251 28285 consensus_queue.cc:260] T 9dd97b6fe0904d62873a10b942fc37c1 P 194dbcf584f04a608c7f6a6a251d71e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "194dbcf584f04a608c7f6a6a251d71e5" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 45915 } } peers { permanent_uuid: "02ad740aae2d470cafcb96d12f589902" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 38259 } } peers { permanent_uuid: "188fbd273bd64a288842883839b5326a" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 42385 } }
I20260806 06:33:35.722317 28065 heartbeater.cc:503] Master 127.7.43.126:40255 was elected leader, sending a full tablet report...
I20260806 06:33:35.722414 28284 consensus_queue.cc:260] T 9dd97b6fe0904d62873a10b942fc37c1 P 188fbd273bd64a288842883839b5326a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "194dbcf584f04a608c7f6a6a251d71e5" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 45915 } } peers { permanent_uuid: "02ad740aae2d470cafcb96d12f589902" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 38259 } } peers { permanent_uuid: "188fbd273bd64a288842883839b5326a" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 42385 } }
I20260806 06:33:35.722529 28285 ts_tablet_manager.cc:1428] T 9dd97b6fe0904d62873a10b942fc37c1 P 194dbcf584f04a608c7f6a6a251d71e5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:33:35.722559 28261 heartbeater.cc:503] Master 127.7.43.126:40255 was elected leader, sending a full tablet report...
I20260806 06:33:35.722908 28284 ts_tablet_manager.cc:1428] T 9dd97b6fe0904d62873a10b942fc37c1 P 188fbd273bd64a288842883839b5326a: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 06:33:35.723065 27886 heartbeater.cc:503] Master 127.7.43.126:40255 was elected leader, sending a full tablet report...
W20260806 06:33:35.788578 28069 tablet.cc:2369] T 9dd97b6fe0904d62873a10b942fc37c1 P 02ad740aae2d470cafcb96d12f589902: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:33:35.799374 28292 raft_consensus.cc:482] T 9dd97b6fe0904d62873a10b942fc37c1 P 188fbd273bd64a288842883839b5326a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:33:35.799484 28292 raft_consensus.cc:504] T 9dd97b6fe0904d62873a10b942fc37c1 P 188fbd273bd64a288842883839b5326a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "194dbcf584f04a608c7f6a6a251d71e5" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 45915 } } peers { permanent_uuid: "02ad740aae2d470cafcb96d12f589902" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 38259 } } peers { permanent_uuid: "188fbd273bd64a288842883839b5326a" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 42385 } }
I20260806 06:33:35.800190 28292 leader_election.cc:288] T 9dd97b6fe0904d62873a10b942fc37c1 P 188fbd273bd64a288842883839b5326a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 194dbcf584f04a608c7f6a6a251d71e5 (127.7.43.67:45915), 02ad740aae2d470cafcb96d12f589902 (127.7.43.66:38259)
I20260806 06:33:35.802557 27888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:35.800031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.67:45915 (local address 127.7.43.65:58091)
0806 06:33:35.800109 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:35.800120 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:33:35.800162 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:33:35.800358 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:33:35.800359 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:33:35.800361 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:33:35.800431 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:35.800433 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:35.801049 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:35.801050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:35.801783 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:35.801785 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:35.802312 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:35.802313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:35.802397 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:35.802401 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:33:35.802446 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 06:33:35.802486 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260806 06:33:35.802557 28276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:35.800094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:58091 (local address 127.7.43.67:45915)
0806 06:33:35.800238 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:35.800240 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:35.800248 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:33:35.800276 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:35.800277 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:35.800289 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:35.800339 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:33:35.800492 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:35.800978 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:35.801870 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:35.802234 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:35.802292 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:35.802305 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:35.802459 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 06:33:35.802496 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260806 06:33:35.802703 28280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:35.800119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:36651 (local address 127.7.43.66:38259)
0806 06:33:35.800165 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:35.800168 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:35.800280 (+ 112us) server_negotiation.cc:407] Connection header received
0806 06:33:35.800374 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:35.800375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:35.800388 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:35.800432 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:33:35.800566 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:35.801056 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:35.801954 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:35.802374 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:35.802438 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:35.802449 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:35.802603 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 06:33:35.802643 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:33:35.802719 28293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:35.800031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.66:38259 (local address 127.7.43.65:36651)
0806 06:33:35.800216 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:35.800226 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:33:35.800309 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:33:35.800433 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:33:35.800434 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:33:35.800436 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:33:35.800510 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:35.800512 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:35.801136 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:35.801137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:35.801875 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:35.801877 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:35.802475 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:35.802476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:35.802545 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:35.802549 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:33:35.802591 (+ 42us) client_negotiation.cc:240] Negotiation successful
0806 06:33:35.802626 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":12,"thread_start_us":115,"threads_started":1}
I20260806 06:33:35.802968 28199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dd97b6fe0904d62873a10b942fc37c1" candidate_uuid: "188fbd273bd64a288842883839b5326a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "194dbcf584f04a608c7f6a6a251d71e5" is_pre_election: true
I20260806 06:33:35.803048 28006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dd97b6fe0904d62873a10b942fc37c1" candidate_uuid: "188fbd273bd64a288842883839b5326a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02ad740aae2d470cafcb96d12f589902" is_pre_election: true
I20260806 06:33:35.803128 28199 raft_consensus.cc:2454] T 9dd97b6fe0904d62873a10b942fc37c1 P 194dbcf584f04a608c7f6a6a251d71e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 188fbd273bd64a288842883839b5326a in term 0.
I20260806 06:33:35.803169 28006 raft_consensus.cc:2454] T 9dd97b6fe0904d62873a10b942fc37c1 P 02ad740aae2d470cafcb96d12f589902 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 188fbd273bd64a288842883839b5326a in term 0.
I20260806 06:33:35.803392 27708 leader_election.cc:302] T 9dd97b6fe0904d62873a10b942fc37c1 P 188fbd273bd64a288842883839b5326a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02ad740aae2d470cafcb96d12f589902, 188fbd273bd64a288842883839b5326a; no voters:
I20260806 06:33:35.803526 28292 raft_consensus.cc:2781] T 9dd97b6fe0904d62873a10b942fc37c1 P 188fbd273bd64a288842883839b5326a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:33:35.803593 28292 raft_consensus.cc:482] T 9dd97b6fe0904d62873a10b942fc37c1 P 188fbd273bd64a288842883839b5326a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:33:35.803609 28292 raft_consensus.cc:3037] T 9dd97b6fe0904d62873a10b942fc37c1 P 188fbd273bd64a288842883839b5326a [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:33:35.804289 28292 raft_consensus.cc:504] T 9dd97b6fe0904d62873a10b942fc37c1 P 188fbd273bd64a288842883839b5326a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "194dbcf584f04a608c7f6a6a251d71e5" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 45915 } } peers { permanent_uuid: "02ad740aae2d470cafcb96d12f589902" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 38259 } } peers { permanent_uuid: "188fbd273bd64a288842883839b5326a" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 42385 } }
I20260806 06:33:35.804404 28292 leader_election.cc:288] T 9dd97b6fe0904d62873a10b942fc37c1 P 188fbd273bd64a288842883839b5326a [CANDIDATE]: Term 1 election: Requested vote from peers 194dbcf584f04a608c7f6a6a251d71e5 (127.7.43.67:45915), 02ad740aae2d470cafcb96d12f589902 (127.7.43.66:38259)
I20260806 06:33:35.804536 28006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dd97b6fe0904d62873a10b942fc37c1" candidate_uuid: "188fbd273bd64a288842883839b5326a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02ad740aae2d470cafcb96d12f589902"
I20260806 06:33:35.804596 28199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dd97b6fe0904d62873a10b942fc37c1" candidate_uuid: "188fbd273bd64a288842883839b5326a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "194dbcf584f04a608c7f6a6a251d71e5"
I20260806 06:33:35.804620 28006 raft_consensus.cc:3037] T 9dd97b6fe0904d62873a10b942fc37c1 P 02ad740aae2d470cafcb96d12f589902 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:33:35.804669 28199 raft_consensus.cc:3037] T 9dd97b6fe0904d62873a10b942fc37c1 P 194dbcf584f04a608c7f6a6a251d71e5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:33:35.805514 28199 raft_consensus.cc:2454] T 9dd97b6fe0904d62873a10b942fc37c1 P 194dbcf584f04a608c7f6a6a251d71e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 188fbd273bd64a288842883839b5326a in term 1.
I20260806 06:33:35.805514 28006 raft_consensus.cc:2454] T 9dd97b6fe0904d62873a10b942fc37c1 P 02ad740aae2d470cafcb96d12f589902 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 188fbd273bd64a288842883839b5326a in term 1.
I20260806 06:33:35.805665 27708 leader_election.cc:302] T 9dd97b6fe0904d62873a10b942fc37c1 P 188fbd273bd64a288842883839b5326a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02ad740aae2d470cafcb96d12f589902, 188fbd273bd64a288842883839b5326a; no voters:
I20260806 06:33:35.805748 28292 raft_consensus.cc:2781] T 9dd97b6fe0904d62873a10b942fc37c1 P 188fbd273bd64a288842883839b5326a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:33:35.805902 28292 raft_consensus.cc:686] T 9dd97b6fe0904d62873a10b942fc37c1 P 188fbd273bd64a288842883839b5326a [term 1 LEADER]: Becoming Leader. State: Replica: 188fbd273bd64a288842883839b5326a, State: Running, Role: LEADER
I20260806 06:33:35.806069 28292 consensus_queue.cc:237] T 9dd97b6fe0904d62873a10b942fc37c1 P 188fbd273bd64a288842883839b5326a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "194dbcf584f04a608c7f6a6a251d71e5" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 45915 } } peers { permanent_uuid: "02ad740aae2d470cafcb96d12f589902" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 38259 } } peers { permanent_uuid: "188fbd273bd64a288842883839b5326a" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 42385 } }
I20260806 06:33:35.806859 27591 catalog_manager.cc:5495] T 9dd97b6fe0904d62873a10b942fc37c1 P 188fbd273bd64a288842883839b5326a reported cstate change: term changed from 0 to 1, leader changed from <none> to 188fbd273bd64a288842883839b5326a (127.7.43.65). New cstate: current_term: 1 leader_uuid: "188fbd273bd64a288842883839b5326a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "194dbcf584f04a608c7f6a6a251d71e5" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 45915 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02ad740aae2d470cafcb96d12f589902" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 38259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "188fbd273bd64a288842883839b5326a" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 42385 } health_report { overall_health: HEALTHY } } }
I20260806 06:33:35.826478 28281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:35.824461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33398 (local address 127.7.43.65:42385)
0806 06:33:35.824519 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:35.824521 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:35.824550 (+ 29us) server_negotiation.cc:407] Connection header received
0806 06:33:35.824576 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:35.824577 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:35.824596 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:35.824643 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:33:35.824792 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:35.825219 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:35.825881 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:35.826046 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:35.826086 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:35.826185 (+ 99us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:33:35.826282 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:33:35.826359 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:35.826375 (+ 16us) server_negotiation.cc:299] Negotiation successful
0806 06:33:35.826394 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 06:33:35.827950 28006 raft_consensus.cc:1261] T 9dd97b6fe0904d62873a10b942fc37c1 P 02ad740aae2d470cafcb96d12f589902 [term 1 FOLLOWER]: Refusing update from remote peer 188fbd273bd64a288842883839b5326a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:33:35.827982 28199 raft_consensus.cc:1261] T 9dd97b6fe0904d62873a10b942fc37c1 P 194dbcf584f04a608c7f6a6a251d71e5 [term 1 FOLLOWER]: Refusing update from remote peer 188fbd273bd64a288842883839b5326a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:33:35.828315 28294 consensus_queue.cc:1035] T 9dd97b6fe0904d62873a10b942fc37c1 P 188fbd273bd64a288842883839b5326a [LEADER]: Connected to new peer: Peer: permanent_uuid: "194dbcf584f04a608c7f6a6a251d71e5" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 45915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:33:35.828431 28292 consensus_queue.cc:1035] T 9dd97b6fe0904d62873a10b942fc37c1 P 188fbd273bd64a288842883839b5326a [LEADER]: Connected to new peer: Peer: permanent_uuid: "02ad740aae2d470cafcb96d12f589902" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 38259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:33:35.830510 28297 mvcc.cc:205] Tried to move back new op lower bound from 7315447872829026304 to 7315447872742207488. Current Snapshot: MvccSnapshot[applied={T|T < 7315447872829026304}]
I20260806 06:33:35.831411 28302 mvcc.cc:205] Tried to move back new op lower bound from 7315447872829026304 to 7315447872742207488. Current Snapshot: MvccSnapshot[applied={T|T < 7315447872829026304}]
I20260806 06:33:35.832597 28299 mvcc.cc:205] Tried to move back new op lower bound from 7315447872829026304 to 7315447872742207488. Current Snapshot: MvccSnapshot[applied={T|T < 7315447872829026304 or (T in {7315447872829026304})}]
I20260806 06:33:35.832825 27590 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53322:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:33:35.832902 27590 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53322:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:33:35.833595 27590 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af: Sending DeleteTablet for 3 replicas of tablet 9dd97b6fe0904d62873a10b942fc37c1
I20260806 06:33:35.833942 27787 tablet_service.cc:1514] Processing DeleteTablet for tablet 9dd97b6fe0904d62873a10b942fc37c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:33:35 UTC) from {username='kudu', principal='kudu/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:42786
I20260806 06:33:35.833962 27977 tablet_service.cc:1514] Processing DeleteTablet for tablet 9dd97b6fe0904d62873a10b942fc37c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:33:35 UTC) from {username='kudu', principal='kudu/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:46124
I20260806 06:33:35.833983 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 27687
I20260806 06:33:35.834151 28170 tablet_service.cc:1514] Processing DeleteTablet for tablet 9dd97b6fe0904d62873a10b942fc37c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:33:35 UTC) from {username='kudu', principal='kudu/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:59570
I20260806 06:33:35.837987 28310 tablet_replica.cc:330] T 9dd97b6fe0904d62873a10b942fc37c1 P 02ad740aae2d470cafcb96d12f589902: stopping tablet replica
I20260806 06:33:35.838080 28310 raft_consensus.cc:2229] T 9dd97b6fe0904d62873a10b942fc37c1 P 02ad740aae2d470cafcb96d12f589902 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:33:35.838167 28310 raft_consensus.cc:2258] T 9dd97b6fe0904d62873a10b942fc37c1 P 02ad740aae2d470cafcb96d12f589902 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:33:35.838590 28310 ts_tablet_manager.cc:1905] T 9dd97b6fe0904d62873a10b942fc37c1 P 02ad740aae2d470cafcb96d12f589902: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:33:35.839795 28310 ts_tablet_manager.cc:1918] T 9dd97b6fe0904d62873a10b942fc37c1 P 02ad740aae2d470cafcb96d12f589902: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:33:35.839844 28310 log.cc:1196] T 9dd97b6fe0904d62873a10b942fc37c1 P 02ad740aae2d470cafcb96d12f589902: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-1/wal/wals/9dd97b6fe0904d62873a10b942fc37c1
I20260806 06:33:35.840018 28310 ts_tablet_manager.cc:1939] T 9dd97b6fe0904d62873a10b942fc37c1 P 02ad740aae2d470cafcb96d12f589902: Deleting consensus metadata
I20260806 06:33:35.840348 27573 catalog_manager.cc:4841] TS 02ad740aae2d470cafcb96d12f589902 (127.7.43.66:38259): tablet 9dd97b6fe0904d62873a10b942fc37c1 (table test-table [id=116800572cf84dc084de49d8c24de0dd]) successfully deleted
I20260806 06:33:35.840492 28312 tablet_replica.cc:330] T 9dd97b6fe0904d62873a10b942fc37c1 P 194dbcf584f04a608c7f6a6a251d71e5: stopping tablet replica
I20260806 06:33:35.840595 28312 raft_consensus.cc:2229] T 9dd97b6fe0904d62873a10b942fc37c1 P 194dbcf584f04a608c7f6a6a251d71e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:33:35.840683 28312 raft_consensus.cc:2258] T 9dd97b6fe0904d62873a10b942fc37c1 P 194dbcf584f04a608c7f6a6a251d71e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:33:35.841055 28312 ts_tablet_manager.cc:1905] T 9dd97b6fe0904d62873a10b942fc37c1 P 194dbcf584f04a608c7f6a6a251d71e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:33:35.842537 28312 ts_tablet_manager.cc:1918] T 9dd97b6fe0904d62873a10b942fc37c1 P 194dbcf584f04a608c7f6a6a251d71e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:33:35.842607 28312 log.cc:1196] T 9dd97b6fe0904d62873a10b942fc37c1 P 194dbcf584f04a608c7f6a6a251d71e5: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-2/wal/wals/9dd97b6fe0904d62873a10b942fc37c1
I20260806 06:33:35.842885 28312 ts_tablet_manager.cc:1939] T 9dd97b6fe0904d62873a10b942fc37c1 P 194dbcf584f04a608c7f6a6a251d71e5: Deleting consensus metadata
W20260806 06:33:35.843004 27576 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.43.65:42385 (error 108)
I20260806 06:33:35.843353 27576 catalog_manager.cc:4841] TS 194dbcf584f04a608c7f6a6a251d71e5 (127.7.43.67:45915): tablet 9dd97b6fe0904d62873a10b942fc37c1 (table test-table [id=116800572cf84dc084de49d8c24de0dd]) successfully deleted
I20260806 06:33:35.843405 28275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:35.843311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:42385 (local address 127.0.0.1:33412)
0806 06:33:35.843347 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:35.843379 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.65:42385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260806 06:33:35.843554 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 27893
W20260806 06:33:35.843554 27576 catalog_manager.cc:4584] TS 188fbd273bd64a288842883839b5326a (127.7.43.65:42385): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9dd97b6fe0904d62873a10b942fc37c1: Network error: Client connection negotiation failed: client connection to 127.7.43.65:42385: connect: Connection refused (error 111)
I20260806 06:33:35.858304 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 28076
I20260806 06:33:35.866236 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 27551
I20260806 06:33:35.871827 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:40255
--webserver_interface=127.7.43.126
--webserver_port=43163
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:35907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.43.126:40255
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:33:35.950861 28314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:33:35.951040 28314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:33:35.951059 28314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:33:35.952122 28314 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:33:35.952153 28314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:33:35.952167 28314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:33:35.952178 28314 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:33:35.952191 28314 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:33:35.953543 28314 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:35907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.7.43.126:40255
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:40255
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.43.126
--webserver_port=43163
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28314
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:33:35.953719 28314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:33:35.953879 28314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:33:35.964408 28325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:33:35.964427 28323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:33:35.964627 28314 server_base.cc:1034] running on GCE node
W20260806 06:33:35.964434 28322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:33:35.965101 28314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:33:35.965373 28314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:33:35.966507 28314 hybrid_clock.cc:648] HybridClock initialized: now 1785998015966493 us; error 30 us; skew 500 ppm
Aug 06 06:33:35 dist-test-slave-6zbq krb5kdc[27531](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998015, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:33:35.968838 28314 init.cc:377] Logged in from keytab as kudu/127.7.43.126@KRBTEST.COM (short username kudu)
I20260806 06:33:35.969220 28314 webserver.cc:460] Webserver started at http://127.7.43.126:43163/ using document root <none> and password file <none>
I20260806 06:33:35.969414 28314 fs_manager.cc:359] Metadata directory not provided
I20260806 06:33:35.969473 28314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:33:35.970770 28314 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:33:35.971318 28338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:35.971459 28314 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:33:35.971540 28314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "6369a1d6b9c64eae826178c1635ec0af"
format_stamp: "Formatted at 2026-08-06 06:33:35 on dist-test-slave-6zbq"
server_key: "d8abb7236e4219fcec9e5c1ce7eb9c77"
server_key_iv: "8008cf321fad7bcd62423025b6735156"
server_key_version: "encryptionkey@0"
I20260806 06:33:35.971891 28314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:33:35.990979 28314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:33:35.992926 28314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:33:35.993052 28314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:33:35.996742 28314 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.126:40255
I20260806 06:33:35.996773 28414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.126:40255 every 8 connection(s)
I20260806 06:33:35.997107 28314 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:33:35.998698 28416 sys_catalog.cc:263] Verifying existing consensus state
I20260806 06:33:35.999357 28416 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af: Bootstrap starting.
I20260806 06:33:36.006431 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 28314
I20260806 06:33:36.006695 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.65:42385
--local_ip_for_outbound_sockets=127.7.43.65
--tserver_master_addrs=127.7.43.126:40255
--webserver_port=42865
--webserver_interface=127.7.43.65
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab
--principal=kudu/127.7.43.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:35907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:33:36.007385 28416 log.cc:824] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af: Log is configured to *not* fsync() on all Append() calls
I20260806 06:33:36.009564 28416 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:33:36.009855 28416 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af: Bootstrap complete.
I20260806 06:33:36.012887 28416 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6369a1d6b9c64eae826178c1635ec0af" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 40255 } }
I20260806 06:33:36.013197 28416 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6369a1d6b9c64eae826178c1635ec0af, State: Initialized, Role: FOLLOWER
I20260806 06:33:36.013368 28416 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6369a1d6b9c64eae826178c1635ec0af" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 40255 } }
I20260806 06:33:36.013443 28416 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:33:36.013497 28416 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:33:36.013547 28416 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af [term 1 FOLLOWER]: Advancing to term 2
I20260806 06:33:36.014896 28416 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6369a1d6b9c64eae826178c1635ec0af" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 40255 } }
I20260806 06:33:36.015010 28416 leader_election.cc:302] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6369a1d6b9c64eae826178c1635ec0af; no voters:
I20260806 06:33:36.015162 28416 leader_election.cc:288] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 06:33:36.015209 28421 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af [term 2 FOLLOWER]: Leader election won for term 2
I20260806 06:33:36.015352 28421 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af [term 2 LEADER]: Becoming Leader. State: Replica: 6369a1d6b9c64eae826178c1635ec0af, State: Running, Role: LEADER
I20260806 06:33:36.015412 28416 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:33:36.015535 28421 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6369a1d6b9c64eae826178c1635ec0af" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 40255 } }
I20260806 06:33:36.015862 28422 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "6369a1d6b9c64eae826178c1635ec0af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6369a1d6b9c64eae826178c1635ec0af" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 40255 } } }
I20260806 06:33:36.015856 28424 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6369a1d6b9c64eae826178c1635ec0af. Latest consensus state: current_term: 2 leader_uuid: "6369a1d6b9c64eae826178c1635ec0af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6369a1d6b9c64eae826178c1635ec0af" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 40255 } } }
I20260806 06:33:36.015957 28422 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af [sys.catalog]: This master's current role is: LEADER
I20260806 06:33:36.015961 28424 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af [sys.catalog]: This master's current role is: LEADER
I20260806 06:33:36.016134 28431 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:33:36.016924 28431 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:33:36.017045 28431 catalog_manager.cc:1259] Loaded cluster ID: c117f87d46394f5fb83dc34a236af890
I20260806 06:33:36.017119 28431 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:33:36.017485 28431 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260806 06:33:36.020328 28431 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260806 06:33:36.021960 28431 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:33:36.022145 28431 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af: Loaded TSK: 0
I20260806 06:33:36.022434 28431 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 06:33:36.102562 28420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:33:36.102751 28420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:33:36.102780 28420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:33:36.103864 28420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:33:36.103904 28420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:33:36.103937 28420 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.65
I20260806 06:33:36.105324 28420 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:35907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab
--principal=kudu/127.7.43.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.65:42385
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.43.65
--webserver_port=42865
--tserver_master_addrs=127.7.43.126:40255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28420
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.65
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:33:36.105543 28420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:33:36.105736 28420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:33:36.110262 28451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:33:36.110256 28455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:33:36.110524 28452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:33:36.110913 28420 server_base.cc:1034] running on GCE node
I20260806 06:33:36.111095 28420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:33:36.111306 28420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:33:36.112450 28420 hybrid_clock.cc:648] HybridClock initialized: now 1785998016112437 us; error 33 us; skew 500 ppm
Aug 06 06:33:36 dist-test-slave-6zbq krb5kdc[27531](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998016, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:33:36.114765 28420 init.cc:377] Logged in from keytab as kudu/127.7.43.65@KRBTEST.COM (short username kudu)
I20260806 06:33:36.115178 28420 webserver.cc:460] Webserver started at http://127.7.43.65:42865/ using document root <none> and password file <none>
I20260806 06:33:36.115387 28420 fs_manager.cc:359] Metadata directory not provided
I20260806 06:33:36.115439 28420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:33:36.116730 28420 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:33:36.117350 28462 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:36.117496 28420 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:36.117573 28420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-0/wal
uuid: "188fbd273bd64a288842883839b5326a"
format_stamp: "Formatted at 2026-08-06 06:33:35 on dist-test-slave-6zbq"
server_key: "92ec15286eb03bb790eb48a65176b51a"
server_key_iv: "54c111ae30fb539743e801216b9f9058"
server_key_version: "encryptionkey@0"
I20260806 06:33:36.117861 28420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:33:36.127332 28420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:33:36.129240 28420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:33:36.129359 28420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:33:36.129539 28420 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:33:36.129964 28474 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 06:33:36.131419 28420 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 06:33:36.131469 28420 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 06:33:36.131490 28420 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 06:33:36.132053 28420 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 06:33:36.132093 28420 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 06:33:36.132167 28474 tablet_bootstrap.cc:492] T 9dd97b6fe0904d62873a10b942fc37c1 P 188fbd273bd64a288842883839b5326a: Bootstrap starting.
I20260806 06:33:36.139426 28420 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.65:42385
I20260806 06:33:36.139899 28420 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
I20260806 06:33:36.140290 28640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.65:42385 every 8 connection(s)
I20260806 06:33:36.142763 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 28420
I20260806 06:33:36.143031 28474 log.cc:824] T 9dd97b6fe0904d62873a10b942fc37c1 P 188fbd273bd64a288842883839b5326a: Log is configured to *not* fsync() on all Append() calls
I20260806 06:33:36.143062 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.66:38259
--local_ip_for_outbound_sockets=127.7.43.66
--tserver_master_addrs=127.7.43.126:40255
--webserver_port=45099
--webserver_interface=127.7.43.66
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab
--principal=kudu/127.7.43.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:35907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:33:36.145524 28474 tablet_bootstrap.cc:492] T 9dd97b6fe0904d62873a10b942fc37c1 P 188fbd273bd64a288842883839b5326a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:33:36.145818 28474 tablet_bootstrap.cc:492] T 9dd97b6fe0904d62873a10b942fc37c1 P 188fbd273bd64a288842883839b5326a: Bootstrap complete.
I20260806 06:33:36.146935 28474 ts_tablet_manager.cc:1397] T 9dd97b6fe0904d62873a10b942fc37c1 P 188fbd273bd64a288842883839b5326a: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.006s
Aug 06 06:33:36 dist-test-slave-6zbq krb5kdc[27531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998016, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.65@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:33:36.148937 28474 raft_consensus.cc:348] T 9dd97b6fe0904d62873a10b942fc37c1 P 188fbd273bd64a288842883839b5326a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "194dbcf584f04a608c7f6a6a251d71e5" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 45915 } } peers { permanent_uuid: "02ad740aae2d470cafcb96d12f589902" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 38259 } } peers { permanent_uuid: "188fbd273bd64a288842883839b5326a" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 42385 } }
I20260806 06:33:36.149310 28474 raft_consensus.cc:729] T 9dd97b6fe0904d62873a10b942fc37c1 P 188fbd273bd64a288842883839b5326a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 188fbd273bd64a288842883839b5326a, State: Initialized, Role: FOLLOWER
I20260806 06:33:36.149484 28474 consensus_queue.cc:260] T 9dd97b6fe0904d62873a10b942fc37c1 P 188fbd273bd64a288842883839b5326a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "194dbcf584f04a608c7f6a6a251d71e5" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 45915 } } peers { permanent_uuid: "02ad740aae2d470cafcb96d12f589902" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 38259 } } peers { permanent_uuid: "188fbd273bd64a288842883839b5326a" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 42385 } }
I20260806 06:33:36.149788 28474 ts_tablet_manager.cc:1428] T 9dd97b6fe0904d62873a10b942fc37c1 P 188fbd273bd64a288842883839b5326a: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260806 06:33:36.153138 28644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:36.140551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.65:60217)
0806 06:33:36.140941 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:36.140983 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 06:33:36.141534 (+ 551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:33:36.142174 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:33:36.142179 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:33:36.142594 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:33:36.142860 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:36.142867 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:36.144540 (+ 1673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:36.144541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:36.145725 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:36.145728 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:36.146564 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:36.146565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:36.146709 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:36.147302 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:33:36.147312 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:33:36.149072 (+ 1760us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:33:36.151521 (+ 2449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:33:36.151527 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:33:36.151530 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:33:36.151798 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:33:36.152053 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:33:36.152054 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:33:36.152054 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:33:36.152176 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:33:36.152571 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:33:36.152577 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:33:36.152708 (+ 131us) client_negotiation.cc:769] Sending connection context
0806 06:33:36.152782 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 06:33:36.152929 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":84,"threads_started":1}
I20260806 06:33:36.153764 28646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:36.140898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:60217 (local address 127.7.43.126:40255)
0806 06:33:36.141721 (+ 823us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:36.141730 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:36.141763 (+ 33us) server_negotiation.cc:407] Connection header received
0806 06:33:36.141884 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:36.141889 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:36.141940 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:36.142064 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:33:36.143008 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:36.144407 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:36.145840 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:36.146437 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:36.146531 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:36.149263 (+ 2732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:33:36.149267 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:33:36.149268 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:33:36.149280 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:33:36.151426 (+ 2146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:36.151882 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:36.151884 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:36.151885 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:36.151987 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:36.152243 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:36.152244 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:36.152244 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:36.152503 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:33:36.153387 (+ 884us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:36.153457 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 06:33:36.153586 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":771,"thread_start_us":96,"threads_started":1}
I20260806 06:33:36.154389 28641 heartbeater.cc:347] Connected to a master server at 127.7.43.126:40255
I20260806 06:33:36.154474 28641 heartbeater.cc:464] Registering TS with master...
I20260806 06:33:36.154642 28641 heartbeater.cc:511] Master 127.7.43.126:40255 requested a full tablet report, sending...
I20260806 06:33:36.155301 28360 ts_manager.cc:194] Registered new tserver with Master: 188fbd273bd64a288842883839b5326a (127.7.43.65:42385)
I20260806 06:33:36.156426 28360 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.65@KRBTEST.COM'} at 127.7.43.65:60217
I20260806 06:33:36.157562 28641 heartbeater.cc:503] Master 127.7.43.126:40255 was elected leader, sending a full tablet report...
I20260806 06:33:36.161815 28652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:36.156811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:42385 (local address 127.0.0.1:33424)
0806 06:33:36.158034 (+ 1223us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:36.158067 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 06:33:36.158133 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:33:36.158626 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:33:36.158628 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:33:36.158631 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:33:36.158689 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:36.158690 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:36.159894 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:36.159895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:36.160760 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:36.160763 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:36.161610 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:36.161610 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:36.161667 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:36.161677 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 06:33:36.161729 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 06:33:36.161767 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1198,"thread_start_us":75,"threads_started":1}
I20260806 06:33:36.161847 28656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:36.156880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33424 (local address 127.7.43.65:42385)
0806 06:33:36.158459 (+ 1579us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:36.158464 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:36.158475 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:33:36.158506 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:36.158508 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:36.158522 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:36.158604 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:33:36.158759 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:36.159801 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:36.160845 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:36.161538 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:36.161617 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:36.161666 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:36.161749 (+ 83us) server_negotiation.cc:299] Negotiation successful
0806 06:33:36.161783 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1548,"thread_start_us":74,"threads_started":1}
I20260806 06:33:36.162164 28548 tablet_service.cc:1514] Processing DeleteTablet for tablet 9dd97b6fe0904d62873a10b942fc37c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:33:35 UTC) from {username='kudu', principal='kudu/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:33424
I20260806 06:33:36.162633 28657 tablet_replica.cc:330] stopping tablet replica
I20260806 06:33:36.162717 28657 raft_consensus.cc:2229] T 9dd97b6fe0904d62873a10b942fc37c1 P 188fbd273bd64a288842883839b5326a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:33:36.162786 28657 raft_consensus.cc:2258] T 9dd97b6fe0904d62873a10b942fc37c1 P 188fbd273bd64a288842883839b5326a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:33:36.163033 28657 ts_tablet_manager.cc:1905] T 9dd97b6fe0904d62873a10b942fc37c1 P 188fbd273bd64a288842883839b5326a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:33:36.164790 28657 ts_tablet_manager.cc:1918] T 9dd97b6fe0904d62873a10b942fc37c1 P 188fbd273bd64a288842883839b5326a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:33:36.164855 28657 log.cc:1196] T 9dd97b6fe0904d62873a10b942fc37c1 P 188fbd273bd64a288842883839b5326a: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-0/wal/wals/9dd97b6fe0904d62873a10b942fc37c1
I20260806 06:33:36.165076 28657 ts_tablet_manager.cc:1939] T 9dd97b6fe0904d62873a10b942fc37c1 P 188fbd273bd64a288842883839b5326a: Deleting consensus metadata
I20260806 06:33:36.165511 28342 catalog_manager.cc:4841] TS 188fbd273bd64a288842883839b5326a (127.7.43.65:42385): tablet 9dd97b6fe0904d62873a10b942fc37c1 (table test-table [id=116800572cf84dc084de49d8c24de0dd]) successfully deleted
W20260806 06:33:36.226868 28649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:33:36.227052 28649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:33:36.227083 28649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:33:36.228173 28649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:33:36.228211 28649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:33:36.228245 28649 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.66
I20260806 06:33:36.229632 28649 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:35907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab
--principal=kudu/127.7.43.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.66:38259
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.43.66
--webserver_port=45099
--tserver_master_addrs=127.7.43.126:40255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28649
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.66
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:33:36.229854 28649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:33:36.230046 28649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:33:36.234612 28669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:33:36.234652 28666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:33:36.234767 28671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:33:36.234767 28649 server_base.cc:1034] running on GCE node
I20260806 06:33:36.235000 28649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:33:36.235213 28649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:33:36.236354 28649 hybrid_clock.cc:648] HybridClock initialized: now 1785998016236339 us; error 27 us; skew 500 ppm
Aug 06 06:33:36 dist-test-slave-6zbq krb5kdc[27531](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998016, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:33:36.238629 28649 init.cc:377] Logged in from keytab as kudu/127.7.43.66@KRBTEST.COM (short username kudu)
I20260806 06:33:36.239006 28649 webserver.cc:460] Webserver started at http://127.7.43.66:45099/ using document root <none> and password file <none>
I20260806 06:33:36.239203 28649 fs_manager.cc:359] Metadata directory not provided
I20260806 06:33:36.239255 28649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:33:36.240464 28649 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:33:36.241044 28678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:36.241226 28649 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 06:33:36.241302 28649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-1/wal
uuid: "02ad740aae2d470cafcb96d12f589902"
format_stamp: "Formatted at 2026-08-06 06:33:35 on dist-test-slave-6zbq"
server_key: "d3c084445a41564da8f73c40ecf31c75"
server_key_iv: "a3eb948d1b8ee2f36fe8d497a026ea36"
server_key_version: "encryptionkey@0"
I20260806 06:33:36.241602 28649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:33:36.254508 28649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:33:36.256403 28649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:33:36.256522 28649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:33:36.256702 28649 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:33:36.256963 28649 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:33:36.256994 28649 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:36.257014 28649 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:33:36.257036 28649 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:36.262606 28649 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.66:38259
I20260806 06:33:36.262643 28837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.66:38259 every 8 connection(s)
I20260806 06:33:36.262951 28649 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
Aug 06 06:33:36 dist-test-slave-6zbq krb5kdc[27531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998016, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.66@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:33:36.270514 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 28649
I20260806 06:33:36.270792 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.67:45915
--local_ip_for_outbound_sockets=127.7.43.67
--tserver_master_addrs=127.7.43.126:40255
--webserver_port=39839
--webserver_interface=127.7.43.67
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab
--principal=kudu/127.7.43.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:35907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:33:36.271596 28646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:36.263548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:34793 (local address 127.7.43.126:40255)
0806 06:33:36.263588 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:36.263590 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:36.263780 (+ 190us) server_negotiation.cc:407] Connection header received
0806 06:33:36.264251 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:36.264253 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:36.264281 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:36.264333 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:33:36.265239 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:36.265727 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:36.266685 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:36.266886 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:36.266943 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:36.268764 (+ 1821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:33:36.268766 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:33:36.268766 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:33:36.268773 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:33:36.270106 (+ 1333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:36.270592 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:36.270593 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:36.270593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:36.270656 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:36.270959 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:36.270961 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:36.270961 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:36.271088 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:33:36.271128 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:36.271462 (+ 334us) server_negotiation.cc:299] Negotiation successful
0806 06:33:36.271532 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:33:36.271778 28841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:36.263528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.66:34793)
0806 06:33:36.263668 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:36.263701 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 06:33:36.264155 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:33:36.264406 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:33:36.264411 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:33:36.264942 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:33:36.265151 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:36.265156 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:36.265813 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:36.265814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:36.266604 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:36.266606 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:36.266956 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:36.266956 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:36.267088 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:36.267442 (+ 354us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:33:36.267449 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:33:36.268668 (+ 1219us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:33:36.270200 (+ 1532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:33:36.270208 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:33:36.270210 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:33:36.270521 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:33:36.270720 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:33:36.270721 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:33:36.270722 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:33:36.270867 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:33:36.271170 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:33:36.271177 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:33:36.271322 (+ 145us) client_negotiation.cc:769] Sending connection context
0806 06:33:36.271425 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 06:33:36.271575 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":65,"threads_started":1}
I20260806 06:33:36.272320 28838 heartbeater.cc:347] Connected to a master server at 127.7.43.126:40255
I20260806 06:33:36.272419 28838 heartbeater.cc:464] Registering TS with master...
I20260806 06:33:36.272603 28838 heartbeater.cc:511] Master 127.7.43.126:40255 requested a full tablet report, sending...
I20260806 06:33:36.272964 28360 ts_manager.cc:194] Registered new tserver with Master: 02ad740aae2d470cafcb96d12f589902 (127.7.43.66:38259)
I20260806 06:33:36.273315 28360 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.66@KRBTEST.COM'} at 127.7.43.66:34793
W20260806 06:33:36.353662 28844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:33:36.353839 28844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:33:36.353869 28844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:33:36.355000 28844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:33:36.355041 28844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:33:36.355073 28844 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.67
I20260806 06:33:36.356479 28844 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:35907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab
--principal=kudu/127.7.43.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.67:45915
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.43.67
--webserver_port=39839
--tserver_master_addrs=127.7.43.126:40255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28844
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.67
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:33:36.356714 28844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:33:36.356897 28844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:33:36.361359 28855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:33:36.361390 28860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:33:36.361371 28856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:33:36.361593 28844 server_base.cc:1034] running on GCE node
I20260806 06:33:36.361766 28844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:33:36.361979 28844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:33:36.363143 28844 hybrid_clock.cc:648] HybridClock initialized: now 1785998016363121 us; error 31 us; skew 500 ppm
Aug 06 06:33:36 dist-test-slave-6zbq krb5kdc[27531](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998016, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:33:36.365460 28844 init.cc:377] Logged in from keytab as kudu/127.7.43.67@KRBTEST.COM (short username kudu)
I20260806 06:33:36.365861 28844 webserver.cc:460] Webserver started at http://127.7.43.67:39839/ using document root <none> and password file <none>
I20260806 06:33:36.366040 28844 fs_manager.cc:359] Metadata directory not provided
I20260806 06:33:36.366089 28844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:33:36.367316 28844 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:33:36.367815 28868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:36.367962 28844 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:36.368040 28844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-2/wal
uuid: "194dbcf584f04a608c7f6a6a251d71e5"
format_stamp: "Formatted at 2026-08-06 06:33:35 on dist-test-slave-6zbq"
server_key: "4d996aa994a70e5b70ed78697afbc27c"
server_key_iv: "0e2e1d01e6bd2bb93a2cedfca6252a31"
server_key_version: "encryptionkey@0"
I20260806 06:33:36.368338 28844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:33:36.390717 28844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:33:36.392634 28844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:33:36.392735 28844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:33:36.392894 28844 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:33:36.393124 28844 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:33:36.393154 28844 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:36.393173 28844 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:33:36.393184 28844 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:36.398501 28844 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.67:45915
I20260806 06:33:36.398564 29037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.67:45915 every 8 connection(s)
I20260806 06:33:36.398856 28844 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
Aug 06 06:33:36 dist-test-slave-6zbq krb5kdc[27531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998016, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.67@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:33:36.406513 28646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:36.399485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:60967 (local address 127.7.43.126:40255)
0806 06:33:36.399558 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:36.399560 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:36.399743 (+ 183us) server_negotiation.cc:407] Connection header received
0806 06:33:36.400189 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:36.400191 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:36.400201 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:36.400238 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:33:36.401052 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:36.401530 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:36.402438 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:36.402614 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:36.402658 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:36.404432 (+ 1774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:33:36.404434 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:33:36.404434 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:33:36.404440 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:33:36.405469 (+ 1029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:36.405810 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:36.405811 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:36.405811 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:36.405853 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:36.406025 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:36.406026 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:36.406026 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:36.406127 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:33:36.406172 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:36.406391 (+ 219us) server_negotiation.cc:299] Negotiation successful
0806 06:33:36.406450 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 06:33:36.406668 29044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:36.399472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.67:60967)
0806 06:33:36.399633 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:36.399665 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 06:33:36.400091 (+ 426us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:33:36.400297 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:33:36.400300 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:33:36.400810 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:33:36.400987 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:36.400992 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:36.401591 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:36.401591 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:36.402363 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:36.402366 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:36.402667 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:36.402668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:36.402760 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:36.403113 (+ 353us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:33:36.403121 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:33:36.404340 (+ 1219us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:33:36.405532 (+ 1192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:33:36.405538 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:33:36.405540 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:33:36.405755 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:33:36.405896 (+ 141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:33:36.405896 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:33:36.405897 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:33:36.405984 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:33:36.406168 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:33:36.406171 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:33:36.406272 (+ 101us) client_negotiation.cc:769] Sending connection context
0806 06:33:36.406372 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 06:33:36.406495 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":89,"threads_started":1}
I20260806 06:33:36.407092 29041 heartbeater.cc:347] Connected to a master server at 127.7.43.126:40255
I20260806 06:33:36.407202 29041 heartbeater.cc:464] Registering TS with master...
I20260806 06:33:36.407364 29041 heartbeater.cc:511] Master 127.7.43.126:40255 requested a full tablet report, sending...
I20260806 06:33:36.407351 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 28844
I20260806 06:33:36.407758 28360 ts_manager.cc:194] Registered new tserver with Master: 194dbcf584f04a608c7f6a6a251d71e5 (127.7.43.67:45915)
I20260806 06:33:36.408177 28360 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.67@KRBTEST.COM'} at 127.7.43.67:60967
I20260806 06:33:36.412747 28646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:36.407699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48868 (local address 127.7.43.126:40255)
0806 06:33:36.407739 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:36.407741 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:36.407868 (+ 127us) server_negotiation.cc:407] Connection header received
0806 06:33:36.407929 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:36.407930 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:36.407939 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:36.407998 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:33:36.408495 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:36.409088 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:36.409735 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:36.409898 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:36.409937 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:36.410896 (+ 959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:33:36.410897 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:33:36.410897 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:33:36.410902 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:33:36.412020 (+ 1118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:36.412283 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:36.412284 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:36.412284 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:36.412324 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:36.412435 (+ 111us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:36.412436 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:36.412436 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:36.412514 (+ 78us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:33:36.412619 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:36.412646 (+ 27us) server_negotiation.cc:299] Negotiation successful
0806 06:33:36.412695 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:33:36.413384 7341 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:33:36.415951 28646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:36.413690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48872 (local address 127.7.43.126:40255)
0806 06:33:36.413739 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:36.413741 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:36.413792 (+ 51us) server_negotiation.cc:407] Connection header received
0806 06:33:36.413870 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:36.413871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:36.413908 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:36.413962 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:33:36.414054 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:36.414533 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:36.415238 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:36.415475 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:36.415527 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:36.415684 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:33:36.415774 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:33:36.415862 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:36.415878 (+ 16us) server_negotiation.cc:299] Negotiation successful
0806 06:33:36.415893 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 06:33:36.416204 28360 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:48872:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:33:36.417742 28360 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:33:36.420248 28548 tablet_service.cc:1467] Processing CreateTablet for tablet abc465d6d67d41e09463dd7c9d9bc276 (DEFAULT_TABLE table=test-table [id=f455aa35c134478388506f2dc4915ca5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:33:36.420466 28548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abc465d6d67d41e09463dd7c9d9bc276. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:33:36.421886 28474 tablet_bootstrap.cc:492] T abc465d6d67d41e09463dd7c9d9bc276 P 188fbd273bd64a288842883839b5326a: Bootstrap starting.
I20260806 06:33:36.422190 28474 tablet_bootstrap.cc:654] T abc465d6d67d41e09463dd7c9d9bc276 P 188fbd273bd64a288842883839b5326a: Neither blocks nor log segments found. Creating new log.
I20260806 06:33:36.422689 28474 tablet_bootstrap.cc:492] T abc465d6d67d41e09463dd7c9d9bc276 P 188fbd273bd64a288842883839b5326a: No bootstrap required, opened a new log
I20260806 06:33:36.422737 28474 ts_tablet_manager.cc:1397] T abc465d6d67d41e09463dd7c9d9bc276 P 188fbd273bd64a288842883839b5326a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:33:36.422835 28474 raft_consensus.cc:348] T abc465d6d67d41e09463dd7c9d9bc276 P 188fbd273bd64a288842883839b5326a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02ad740aae2d470cafcb96d12f589902" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 38259 } } peers { permanent_uuid: "194dbcf584f04a608c7f6a6a251d71e5" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 45915 } } peers { permanent_uuid: "188fbd273bd64a288842883839b5326a" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 42385 } }
I20260806 06:33:36.422895 28474 raft_consensus.cc:374] T abc465d6d67d41e09463dd7c9d9bc276 P 188fbd273bd64a288842883839b5326a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:33:36.422931 28474 raft_consensus.cc:729] T abc465d6d67d41e09463dd7c9d9bc276 P 188fbd273bd64a288842883839b5326a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 188fbd273bd64a288842883839b5326a, State: Initialized, Role: FOLLOWER
I20260806 06:33:36.422973 28474 consensus_queue.cc:260] T abc465d6d67d41e09463dd7c9d9bc276 P 188fbd273bd64a288842883839b5326a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02ad740aae2d470cafcb96d12f589902" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 38259 } } peers { permanent_uuid: "194dbcf584f04a608c7f6a6a251d71e5" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 45915 } } peers { permanent_uuid: "188fbd273bd64a288842883839b5326a" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 42385 } }
I20260806 06:33:36.423056 28474 ts_tablet_manager.cc:1428] T abc465d6d67d41e09463dd7c9d9bc276 P 188fbd273bd64a288842883839b5326a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:36.423159 28652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:36.419436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.66:38259 (local address 127.0.0.1:35264)
0806 06:33:36.419511 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:36.419523 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:33:36.419567 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:33:36.420044 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:33:36.420045 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:33:36.420047 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:33:36.420082 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:36.420083 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:36.421077 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:36.421078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:36.421975 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:36.421977 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:36.422953 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:36.422954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:36.423009 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:36.423012 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:33:36.423058 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 06:33:36.423102 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260806 06:33:36.423434 29048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:36.419739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35264 (local address 127.7.43.66:38259)
0806 06:33:36.419893 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:36.419898 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:36.419907 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:33:36.419948 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:36.419950 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:36.419965 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:36.420047 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:33:36.420139 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:36.420971 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:36.422252 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:36.422879 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:36.422941 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:36.422996 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:36.423311 (+ 315us) server_negotiation.cc:299] Negotiation successful
0806 06:33:36.423356 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":87,"threads_started":1}
I20260806 06:33:36.423902 29050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:36.419460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.67:45915 (local address 127.0.0.1:52644)
0806 06:33:36.420005 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:36.420014 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:33:36.420061 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:33:36.420294 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:33:36.420294 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:33:36.420296 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:33:36.420335 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:36.420336 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:36.421511 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:36.421512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:36.422534 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:36.422537 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:36.423452 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:36.423453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:36.423733 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:36.423738 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:33:36.423795 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 06:33:36.423835 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"mutex_wait_us":343,"thread_start_us":45,"threads_started":1}
I20260806 06:33:36.423945 29047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:36.419661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52644 (local address 127.7.43.67:45915)
0806 06:33:36.419818 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:36.419823 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:36.420113 (+ 290us) server_negotiation.cc:407] Connection header received
0806 06:33:36.420150 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:36.420152 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:36.420166 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:36.420228 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:33:36.420406 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:36.421402 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:36.422626 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:36.423281 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:36.423345 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:36.423391 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:36.423828 (+ 437us) server_negotiation.cc:299] Negotiation successful
0806 06:33:36.423871 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":86,"threads_started":1}
I20260806 06:33:36.424118 28745 tablet_service.cc:1467] Processing CreateTablet for tablet abc465d6d67d41e09463dd7c9d9bc276 (DEFAULT_TABLE table=test-table [id=f455aa35c134478388506f2dc4915ca5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:33:36.424428 28745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abc465d6d67d41e09463dd7c9d9bc276. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:33:36.424489 28940 tablet_service.cc:1467] Processing CreateTablet for tablet abc465d6d67d41e09463dd7c9d9bc276 (DEFAULT_TABLE table=test-table [id=f455aa35c134478388506f2dc4915ca5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:33:36.424762 28940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abc465d6d67d41e09463dd7c9d9bc276. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:33:36.427088 29052 tablet_bootstrap.cc:492] T abc465d6d67d41e09463dd7c9d9bc276 P 02ad740aae2d470cafcb96d12f589902: Bootstrap starting.
I20260806 06:33:36.427243 29053 tablet_bootstrap.cc:492] T abc465d6d67d41e09463dd7c9d9bc276 P 194dbcf584f04a608c7f6a6a251d71e5: Bootstrap starting.
I20260806 06:33:36.427664 29052 tablet_bootstrap.cc:654] T abc465d6d67d41e09463dd7c9d9bc276 P 02ad740aae2d470cafcb96d12f589902: Neither blocks nor log segments found. Creating new log.
I20260806 06:33:36.427757 29053 tablet_bootstrap.cc:654] T abc465d6d67d41e09463dd7c9d9bc276 P 194dbcf584f04a608c7f6a6a251d71e5: Neither blocks nor log segments found. Creating new log.
I20260806 06:33:36.428172 29052 log.cc:824] T abc465d6d67d41e09463dd7c9d9bc276 P 02ad740aae2d470cafcb96d12f589902: Log is configured to *not* fsync() on all Append() calls
I20260806 06:33:36.428331 29053 log.cc:824] T abc465d6d67d41e09463dd7c9d9bc276 P 194dbcf584f04a608c7f6a6a251d71e5: Log is configured to *not* fsync() on all Append() calls
I20260806 06:33:36.429193 29053 tablet_bootstrap.cc:492] T abc465d6d67d41e09463dd7c9d9bc276 P 194dbcf584f04a608c7f6a6a251d71e5: No bootstrap required, opened a new log
I20260806 06:33:36.429271 29053 ts_tablet_manager.cc:1397] T abc465d6d67d41e09463dd7c9d9bc276 P 194dbcf584f04a608c7f6a6a251d71e5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:33:36.429880 29052 tablet_bootstrap.cc:492] T abc465d6d67d41e09463dd7c9d9bc276 P 02ad740aae2d470cafcb96d12f589902: No bootstrap required, opened a new log
I20260806 06:33:36.429950 29052 ts_tablet_manager.cc:1397] T abc465d6d67d41e09463dd7c9d9bc276 P 02ad740aae2d470cafcb96d12f589902: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 06:33:36.431084 29053 raft_consensus.cc:348] T abc465d6d67d41e09463dd7c9d9bc276 P 194dbcf584f04a608c7f6a6a251d71e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02ad740aae2d470cafcb96d12f589902" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 38259 } } peers { permanent_uuid: "194dbcf584f04a608c7f6a6a251d71e5" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 45915 } } peers { permanent_uuid: "188fbd273bd64a288842883839b5326a" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 42385 } }
I20260806 06:33:36.431223 29053 raft_consensus.cc:374] T abc465d6d67d41e09463dd7c9d9bc276 P 194dbcf584f04a608c7f6a6a251d71e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:33:36.431260 29053 raft_consensus.cc:729] T abc465d6d67d41e09463dd7c9d9bc276 P 194dbcf584f04a608c7f6a6a251d71e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 194dbcf584f04a608c7f6a6a251d71e5, State: Initialized, Role: FOLLOWER
I20260806 06:33:36.431368 29053 consensus_queue.cc:260] T abc465d6d67d41e09463dd7c9d9bc276 P 194dbcf584f04a608c7f6a6a251d71e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02ad740aae2d470cafcb96d12f589902" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 38259 } } peers { permanent_uuid: "194dbcf584f04a608c7f6a6a251d71e5" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 45915 } } peers { permanent_uuid: "188fbd273bd64a288842883839b5326a" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 42385 } }
I20260806 06:33:36.431628 29053 ts_tablet_manager.cc:1428] T abc465d6d67d41e09463dd7c9d9bc276 P 194dbcf584f04a608c7f6a6a251d71e5: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 06:33:36.431656 29041 heartbeater.cc:503] Master 127.7.43.126:40255 was elected leader, sending a full tablet report...
I20260806 06:33:36.431787 29052 raft_consensus.cc:348] T abc465d6d67d41e09463dd7c9d9bc276 P 02ad740aae2d470cafcb96d12f589902 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02ad740aae2d470cafcb96d12f589902" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 38259 } } peers { permanent_uuid: "194dbcf584f04a608c7f6a6a251d71e5" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 45915 } } peers { permanent_uuid: "188fbd273bd64a288842883839b5326a" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 42385 } }
I20260806 06:33:36.431947 29052 raft_consensus.cc:374] T abc465d6d67d41e09463dd7c9d9bc276 P 02ad740aae2d470cafcb96d12f589902 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:33:36.431991 29052 raft_consensus.cc:729] T abc465d6d67d41e09463dd7c9d9bc276 P 02ad740aae2d470cafcb96d12f589902 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02ad740aae2d470cafcb96d12f589902, State: Initialized, Role: FOLLOWER
I20260806 06:33:36.432098 29052 consensus_queue.cc:260] T abc465d6d67d41e09463dd7c9d9bc276 P 02ad740aae2d470cafcb96d12f589902 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02ad740aae2d470cafcb96d12f589902" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 38259 } } peers { permanent_uuid: "194dbcf584f04a608c7f6a6a251d71e5" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 45915 } } peers { permanent_uuid: "188fbd273bd64a288842883839b5326a" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 42385 } }
I20260806 06:33:36.432368 29052 ts_tablet_manager.cc:1428] T abc465d6d67d41e09463dd7c9d9bc276 P 02ad740aae2d470cafcb96d12f589902: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 06:33:36.432395 28838 heartbeater.cc:503] Master 127.7.43.126:40255 was elected leader, sending a full tablet report...
I20260806 06:33:36.484191 29056 raft_consensus.cc:482] T abc465d6d67d41e09463dd7c9d9bc276 P 194dbcf584f04a608c7f6a6a251d71e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:33:36.484287 29056 raft_consensus.cc:504] T abc465d6d67d41e09463dd7c9d9bc276 P 194dbcf584f04a608c7f6a6a251d71e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02ad740aae2d470cafcb96d12f589902" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 38259 } } peers { permanent_uuid: "194dbcf584f04a608c7f6a6a251d71e5" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 45915 } } peers { permanent_uuid: "188fbd273bd64a288842883839b5326a" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 42385 } }
I20260806 06:33:36.484584 29056 leader_election.cc:288] T abc465d6d67d41e09463dd7c9d9bc276 P 194dbcf584f04a608c7f6a6a251d71e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02ad740aae2d470cafcb96d12f589902 (127.7.43.66:38259), 188fbd273bd64a288842883839b5326a (127.7.43.65:42385)
I20260806 06:33:36.487190 29044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:36.484729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.66:38259 (local address 127.7.43.67:59043)
0806 06:33:36.484805 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:36.484816 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:33:36.484854 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:33:36.485014 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:33:36.485015 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:33:36.485016 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:33:36.485068 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:36.485069 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:36.485626 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:36.485627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:36.486334 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:36.486363 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:36.486948 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:36.486949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:36.487027 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:36.487030 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:33:36.487087 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 06:33:36.487123 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260806 06:33:36.487190 29048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:36.484841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:59043 (local address 127.7.43.66:38259)
0806 06:33:36.484916 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:36.484918 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:36.484925 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:33:36.484942 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:36.484943 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:36.484952 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:36.484996 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:33:36.485119 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:36.485555 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:36.486450 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:36.486877 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:36.486929 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:36.486937 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:36.487097 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 06:33:36.487130 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 06:33:36.487485 29058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:36.484729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:42385 (local address 127.7.43.67:44979)
0806 06:33:36.484902 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:36.484912 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:33:36.484976 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:33:36.485101 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:33:36.485102 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:33:36.485103 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:33:36.485168 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:36.485170 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:36.485747 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:36.485748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:36.486551 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:36.486554 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:36.487112 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:36.487113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:36.487180 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:36.487184 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:33:36.487367 (+ 183us) client_negotiation.cc:240] Negotiation successful
0806 06:33:36.487406 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":12,"thread_start_us":102,"threads_started":1}
I20260806 06:33:36.487497 28656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:36.484788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:44979 (local address 127.7.43.65:42385)
0806 06:33:36.484885 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:36.484887 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:36.484974 (+ 87us) server_negotiation.cc:407] Connection header received
0806 06:33:36.485025 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:36.485026 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:36.485037 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:36.485089 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:33:36.485225 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:36.485680 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:36.486623 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:36.487024 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:36.487094 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:36.487104 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:36.487388 (+ 284us) server_negotiation.cc:299] Negotiation successful
0806 06:33:36.487438 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 06:33:36.487494 28768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abc465d6d67d41e09463dd7c9d9bc276" candidate_uuid: "194dbcf584f04a608c7f6a6a251d71e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02ad740aae2d470cafcb96d12f589902" is_pre_election: true
I20260806 06:33:36.487608 28768 raft_consensus.cc:2454] T abc465d6d67d41e09463dd7c9d9bc276 P 02ad740aae2d470cafcb96d12f589902 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 194dbcf584f04a608c7f6a6a251d71e5 in term 0.
I20260806 06:33:36.487814 28869 leader_election.cc:302] T abc465d6d67d41e09463dd7c9d9bc276 P 194dbcf584f04a608c7f6a6a251d71e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02ad740aae2d470cafcb96d12f589902, 194dbcf584f04a608c7f6a6a251d71e5; no voters:
I20260806 06:33:36.487811 28574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abc465d6d67d41e09463dd7c9d9bc276" candidate_uuid: "194dbcf584f04a608c7f6a6a251d71e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "188fbd273bd64a288842883839b5326a" is_pre_election: true
I20260806 06:33:36.487924 29056 raft_consensus.cc:2781] T abc465d6d67d41e09463dd7c9d9bc276 P 194dbcf584f04a608c7f6a6a251d71e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:33:36.487934 28574 raft_consensus.cc:2454] T abc465d6d67d41e09463dd7c9d9bc276 P 188fbd273bd64a288842883839b5326a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 194dbcf584f04a608c7f6a6a251d71e5 in term 0.
I20260806 06:33:36.487970 29056 raft_consensus.cc:482] T abc465d6d67d41e09463dd7c9d9bc276 P 194dbcf584f04a608c7f6a6a251d71e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:33:36.487985 29056 raft_consensus.cc:3037] T abc465d6d67d41e09463dd7c9d9bc276 P 194dbcf584f04a608c7f6a6a251d71e5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:33:36.488588 29056 raft_consensus.cc:504] T abc465d6d67d41e09463dd7c9d9bc276 P 194dbcf584f04a608c7f6a6a251d71e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02ad740aae2d470cafcb96d12f589902" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 38259 } } peers { permanent_uuid: "194dbcf584f04a608c7f6a6a251d71e5" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 45915 } } peers { permanent_uuid: "188fbd273bd64a288842883839b5326a" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 42385 } }
I20260806 06:33:36.488729 29056 leader_election.cc:288] T abc465d6d67d41e09463dd7c9d9bc276 P 194dbcf584f04a608c7f6a6a251d71e5 [CANDIDATE]: Term 1 election: Requested vote from peers 02ad740aae2d470cafcb96d12f589902 (127.7.43.66:38259), 188fbd273bd64a288842883839b5326a (127.7.43.65:42385)
I20260806 06:33:36.488888 28574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abc465d6d67d41e09463dd7c9d9bc276" candidate_uuid: "194dbcf584f04a608c7f6a6a251d71e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "188fbd273bd64a288842883839b5326a"
I20260806 06:33:36.488945 28768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abc465d6d67d41e09463dd7c9d9bc276" candidate_uuid: "194dbcf584f04a608c7f6a6a251d71e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02ad740aae2d470cafcb96d12f589902"
I20260806 06:33:36.488961 28574 raft_consensus.cc:3037] T abc465d6d67d41e09463dd7c9d9bc276 P 188fbd273bd64a288842883839b5326a [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:33:36.489017 28768 raft_consensus.cc:3037] T abc465d6d67d41e09463dd7c9d9bc276 P 02ad740aae2d470cafcb96d12f589902 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:33:36.489635 28768 raft_consensus.cc:2454] T abc465d6d67d41e09463dd7c9d9bc276 P 02ad740aae2d470cafcb96d12f589902 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 194dbcf584f04a608c7f6a6a251d71e5 in term 1.
I20260806 06:33:36.489784 28869 leader_election.cc:302] T abc465d6d67d41e09463dd7c9d9bc276 P 194dbcf584f04a608c7f6a6a251d71e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02ad740aae2d470cafcb96d12f589902, 194dbcf584f04a608c7f6a6a251d71e5; no voters:
I20260806 06:33:36.489861 29056 raft_consensus.cc:2781] T abc465d6d67d41e09463dd7c9d9bc276 P 194dbcf584f04a608c7f6a6a251d71e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:33:36.489967 28574 raft_consensus.cc:2454] T abc465d6d67d41e09463dd7c9d9bc276 P 188fbd273bd64a288842883839b5326a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 194dbcf584f04a608c7f6a6a251d71e5 in term 1.
I20260806 06:33:36.489979 29056 raft_consensus.cc:686] T abc465d6d67d41e09463dd7c9d9bc276 P 194dbcf584f04a608c7f6a6a251d71e5 [term 1 LEADER]: Becoming Leader. State: Replica: 194dbcf584f04a608c7f6a6a251d71e5, State: Running, Role: LEADER
I20260806 06:33:36.490059 29056 consensus_queue.cc:237] T abc465d6d67d41e09463dd7c9d9bc276 P 194dbcf584f04a608c7f6a6a251d71e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02ad740aae2d470cafcb96d12f589902" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 38259 } } peers { permanent_uuid: "194dbcf584f04a608c7f6a6a251d71e5" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 45915 } } peers { permanent_uuid: "188fbd273bd64a288842883839b5326a" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 42385 } }
I20260806 06:33:36.490787 28360 catalog_manager.cc:5495] T abc465d6d67d41e09463dd7c9d9bc276 P 194dbcf584f04a608c7f6a6a251d71e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 194dbcf584f04a608c7f6a6a251d71e5 (127.7.43.67). New cstate: current_term: 1 leader_uuid: "194dbcf584f04a608c7f6a6a251d71e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02ad740aae2d470cafcb96d12f589902" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 38259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "194dbcf584f04a608c7f6a6a251d71e5" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 45915 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "188fbd273bd64a288842883839b5326a" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 42385 } health_report { overall_health: UNKNOWN } } }
I20260806 06:33:36.502220 29047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:36.500446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52646 (local address 127.7.43.67:45915)
0806 06:33:36.500481 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:36.500482 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:36.500491 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:33:36.500551 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:36.500552 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:36.500566 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:36.500618 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:33:36.500695 (+ 77us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:36.501116 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:36.501719 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:36.501881 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:36.501921 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:36.502012 (+ 91us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:33:36.502086 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:33:36.502134 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:36.502148 (+ 14us) server_negotiation.cc:299] Negotiation successful
0806 06:33:36.502161 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 06:33:36.503726 28574 raft_consensus.cc:1261] T abc465d6d67d41e09463dd7c9d9bc276 P 188fbd273bd64a288842883839b5326a [term 1 FOLLOWER]: Refusing update from remote peer 194dbcf584f04a608c7f6a6a251d71e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:33:36.503722 28768 raft_consensus.cc:1261] T abc465d6d67d41e09463dd7c9d9bc276 P 02ad740aae2d470cafcb96d12f589902 [term 1 FOLLOWER]: Refusing update from remote peer 194dbcf584f04a608c7f6a6a251d71e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:33:36.504017 29056 consensus_queue.cc:1035] T abc465d6d67d41e09463dd7c9d9bc276 P 194dbcf584f04a608c7f6a6a251d71e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "188fbd273bd64a288842883839b5326a" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 42385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:33:36.504163 29059 consensus_queue.cc:1035] T abc465d6d67d41e09463dd7c9d9bc276 P 194dbcf584f04a608c7f6a6a251d71e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "02ad740aae2d470cafcb96d12f589902" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 38259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:33:36.505923 29062 mvcc.cc:205] Tried to move back new op lower bound from 7315447875597008896 to 7315447875543859200. Current Snapshot: MvccSnapshot[applied={T|T < 7315447875597008896}]
I20260806 06:33:36.506537 29065 mvcc.cc:205] Tried to move back new op lower bound from 7315447875597008896 to 7315447875543859200. Current Snapshot: MvccSnapshot[applied={T|T < 7315447875597008896}]
I20260806 06:33:36.506718 29063 mvcc.cc:205] Tried to move back new op lower bound from 7315447875597008896 to 7315447875543859200. Current Snapshot: MvccSnapshot[applied={T|T < 7315447875597008896}]
I20260806 06:33:36.508196 28360 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:48872:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:33:36.508266 28360 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:48872:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:33:36.508833 28360 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af: Sending DeleteTablet for 3 replicas of tablet abc465d6d67d41e09463dd7c9d9bc276
I20260806 06:33:36.509160 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 28420
I20260806 06:33:36.509189 28940 tablet_service.cc:1514] Processing DeleteTablet for tablet abc465d6d67d41e09463dd7c9d9bc276 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:33:36 UTC) from {username='kudu', principal='kudu/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:52644
I20260806 06:33:36.509172 28745 tablet_service.cc:1514] Processing DeleteTablet for tablet abc465d6d67d41e09463dd7c9d9bc276 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:33:36 UTC) from {username='kudu', principal='kudu/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:35264
I20260806 06:33:36.509338 28548 tablet_service.cc:1514] Processing DeleteTablet for tablet abc465d6d67d41e09463dd7c9d9bc276 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:33:36 UTC) from {username='kudu', principal='kudu/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:33424
W20260806 06:33:36.521202 28342 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.43.65:42385 (error 108)
I20260806 06:33:36.521485 29077 tablet_replica.cc:330] T abc465d6d67d41e09463dd7c9d9bc276 P 194dbcf584f04a608c7f6a6a251d71e5: stopping tablet replica
I20260806 06:33:36.521629 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 28649
I20260806 06:33:36.521680 29050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:36.521561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:42385 (local address 127.0.0.1:33428)
0806 06:33:36.521609 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:36.521652 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.65:42385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260806 06:33:36.521692 29076 tablet_replica.cc:330] T abc465d6d67d41e09463dd7c9d9bc276 P 02ad740aae2d470cafcb96d12f589902: stopping tablet replica
I20260806 06:33:36.521788 29076 raft_consensus.cc:2229] T abc465d6d67d41e09463dd7c9d9bc276 P 02ad740aae2d470cafcb96d12f589902 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:33:36.521883 29076 raft_consensus.cc:2258] T abc465d6d67d41e09463dd7c9d9bc276 P 02ad740aae2d470cafcb96d12f589902 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260806 06:33:36.521786 28342 catalog_manager.cc:4584] TS 188fbd273bd64a288842883839b5326a (127.7.43.65:42385): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet abc465d6d67d41e09463dd7c9d9bc276: Network error: Client connection negotiation failed: client connection to 127.7.43.65:42385: connect: Connection refused (error 111)
I20260806 06:33:36.522493 29077 raft_consensus.cc:2229] T abc465d6d67d41e09463dd7c9d9bc276 P 194dbcf584f04a608c7f6a6a251d71e5 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:33:36.523335 29077 raft_consensus.cc:2258] T abc465d6d67d41e09463dd7c9d9bc276 P 194dbcf584f04a608c7f6a6a251d71e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:33:36.523782 29077 ts_tablet_manager.cc:1905] T abc465d6d67d41e09463dd7c9d9bc276 P 194dbcf584f04a608c7f6a6a251d71e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:33:36.526428 29077 ts_tablet_manager.cc:1918] T abc465d6d67d41e09463dd7c9d9bc276 P 194dbcf584f04a608c7f6a6a251d71e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:33:36.526496 29077 log.cc:1196] T abc465d6d67d41e09463dd7c9d9bc276 P 194dbcf584f04a608c7f6a6a251d71e5: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-2/wal/wals/abc465d6d67d41e09463dd7c9d9bc276
I20260806 06:33:36.526736 29077 ts_tablet_manager.cc:1939] T abc465d6d67d41e09463dd7c9d9bc276 P 194dbcf584f04a608c7f6a6a251d71e5: Deleting consensus metadata
I20260806 06:33:36.527096 28342 catalog_manager.cc:4841] TS 194dbcf584f04a608c7f6a6a251d71e5 (127.7.43.67:45915): tablet abc465d6d67d41e09463dd7c9d9bc276 (table test-table [id=f455aa35c134478388506f2dc4915ca5]) successfully deleted
I20260806 06:33:36.529776 29050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:36.529672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.66:38259 (local address 127.0.0.1:35272)
0806 06:33:36.529720 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:36.529748 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.66:38259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 06:33:36.529848 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 28844
I20260806 06:33:36.538738 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 28314
I20260806 06:33:36.545897 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:40255
--webserver_interface=127.7.43.126
--webserver_port=43163
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:35907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.43.126:40255
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:33:36.625658 29080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:33:36.625845 29080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:33:36.625874 29080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:33:36.627053 29080 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:33:36.627102 29080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:33:36.627122 29080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:33:36.627138 29080 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:33:36.627156 29080 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:33:36.628603 29080 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:35907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.43.126:40255
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:40255
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.43.126
--webserver_port=43163
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29080
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:33:36.628835 29080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:33:36.629025 29080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:33:36.633641 29086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:33:36.633626 29089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:33:36.633772 29080 server_base.cc:1034] running on GCE node
W20260806 06:33:36.633620 29085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:33:36.634063 29080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:33:36.634330 29080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:33:36.635501 29080 hybrid_clock.cc:648] HybridClock initialized: now 1785998016635483 us; error 35 us; skew 500 ppm
Aug 06 06:33:36 dist-test-slave-6zbq krb5kdc[27531](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998016, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:33:36.637773 29080 init.cc:377] Logged in from keytab as kudu/127.7.43.126@KRBTEST.COM (short username kudu)
I20260806 06:33:36.638173 29080 webserver.cc:460] Webserver started at http://127.7.43.126:43163/ using document root <none> and password file <none>
I20260806 06:33:36.638427 29080 fs_manager.cc:359] Metadata directory not provided
I20260806 06:33:36.638473 29080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:33:36.639652 29080 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:33:36.640218 29103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:36.640340 29080 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:36.640415 29080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "6369a1d6b9c64eae826178c1635ec0af"
format_stamp: "Formatted at 2026-08-06 06:33:35 on dist-test-slave-6zbq"
server_key: "d8abb7236e4219fcec9e5c1ce7eb9c77"
server_key_iv: "8008cf321fad7bcd62423025b6735156"
server_key_version: "encryptionkey@0"
I20260806 06:33:36.640713 29080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:33:36.650941 29080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:33:36.652784 29080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:33:36.652900 29080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:33:36.656461 29080 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.126:40255
I20260806 06:33:36.656492 29181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.126:40255 every 8 connection(s)
I20260806 06:33:36.656803 29080 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:33:36.658174 29183 sys_catalog.cc:263] Verifying existing consensus state
I20260806 06:33:36.658772 29183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af: Bootstrap starting.
I20260806 06:33:36.660355 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 29080
I20260806 06:33:36.660678 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.65:42385
--local_ip_for_outbound_sockets=127.7.43.65
--tserver_master_addrs=127.7.43.126:40255
--webserver_port=42865
--webserver_interface=127.7.43.65
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab
--principal=kudu/127.7.43.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:35907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:33:36.667297 29183 log.cc:824] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af: Log is configured to *not* fsync() on all Append() calls
I20260806 06:33:36.669252 29183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:33:36.669497 29183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af: Bootstrap complete.
I20260806 06:33:36.671490 29183 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6369a1d6b9c64eae826178c1635ec0af" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 40255 } }
I20260806 06:33:36.671700 29183 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6369a1d6b9c64eae826178c1635ec0af, State: Initialized, Role: FOLLOWER
I20260806 06:33:36.671851 29183 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6369a1d6b9c64eae826178c1635ec0af" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 40255 } }
I20260806 06:33:36.671909 29183 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:33:36.671954 29183 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:33:36.671998 29183 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af [term 2 FOLLOWER]: Advancing to term 3
I20260806 06:33:36.672892 29183 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6369a1d6b9c64eae826178c1635ec0af" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 40255 } }
I20260806 06:33:36.672986 29183 leader_election.cc:302] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6369a1d6b9c64eae826178c1635ec0af; no voters:
I20260806 06:33:36.673130 29183 leader_election.cc:288] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 06:33:36.673199 29187 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af [term 3 FOLLOWER]: Leader election won for term 3
I20260806 06:33:36.673373 29187 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af [term 3 LEADER]: Becoming Leader. State: Replica: 6369a1d6b9c64eae826178c1635ec0af, State: Running, Role: LEADER
I20260806 06:33:36.673380 29183 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:33:36.673537 29187 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6369a1d6b9c64eae826178c1635ec0af" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 40255 } }
I20260806 06:33:36.673830 29189 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "6369a1d6b9c64eae826178c1635ec0af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6369a1d6b9c64eae826178c1635ec0af" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 40255 } } }
I20260806 06:33:36.673913 29189 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af [sys.catalog]: This master's current role is: LEADER
I20260806 06:33:36.674217 29192 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6369a1d6b9c64eae826178c1635ec0af. Latest consensus state: current_term: 3 leader_uuid: "6369a1d6b9c64eae826178c1635ec0af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6369a1d6b9c64eae826178c1635ec0af" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 40255 } } }
I20260806 06:33:36.674284 29192 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af [sys.catalog]: This master's current role is: LEADER
I20260806 06:33:36.675179 29214 catalog_manager.cc:1259] Loaded cluster ID: c117f87d46394f5fb83dc34a236af890
I20260806 06:33:36.675235 29214 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af: loading cluster ID for follower catalog manager: success
W20260806 06:33:36.675529 29214 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 6369a1d6b9c64eae826178c1635ec0af: acquiring CA information for follower catalog manager: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
W20260806 06:33:36.675580 29214 catalog_manager.cc:873] Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO: failed to prepare follower catalog manager, will retry
I20260806 06:33:36.675611 29200 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:33:36.675990 29200 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:33:36.676079 29200 catalog_manager.cc:1259] Loaded cluster ID: c117f87d46394f5fb83dc34a236af890
I20260806 06:33:36.676111 29200 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260806 06:33:36.676294 29200 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f75453f89ad google::LogMessage::Fail() at ??:0
@ 0x7f75453fb88a google::LogMessage::SendToLog() at ??:0
@ 0x7f75453f84de google::LogMessage::Flush() at ??:0
@ 0x7f75453f9a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f754d3190f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f754d3228b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f75472c60d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f75472bb954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f754811f6db start_thread at ??:0
@ 0x7f7543ef661f clone at ??:0
W20260806 06:33:36.751928 29185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:33:36.752105 29185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:33:36.752133 29185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:33:36.753217 29185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:33:36.753254 29185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:33:36.753278 29185 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.65
I20260806 06:33:36.754691 29185 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:35907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab
--principal=kudu/127.7.43.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.65:42385
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.43.65
--webserver_port=42865
--tserver_master_addrs=127.7.43.126:40255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29185
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.65
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:33:36.754887 29185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:33:36.755084 29185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:33:36.759720 29243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:33:36.759729 29245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:33:36.759891 29185 server_base.cc:1034] running on GCE node
W20260806 06:33:36.760000 29241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:33:36.760215 29185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:33:36.760474 29185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:33:36.761658 29185 hybrid_clock.cc:648] HybridClock initialized: now 1785998016761633 us; error 38 us; skew 500 ppm
Aug 06 06:33:36 dist-test-slave-6zbq krb5kdc[27531](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998016, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:33:36.764459 29185 init.cc:377] Logged in from keytab as kudu/127.7.43.65@KRBTEST.COM (short username kudu)
I20260806 06:33:36.764972 29185 webserver.cc:460] Webserver started at http://127.7.43.65:42865/ using document root <none> and password file <none>
I20260806 06:33:36.765223 29185 fs_manager.cc:359] Metadata directory not provided
I20260806 06:33:36.765286 29185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:33:36.767026 29185 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:33:36.767830 29255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:36.768003 29185 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:33:36.768101 29185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-0/wal
uuid: "188fbd273bd64a288842883839b5326a"
format_stamp: "Formatted at 2026-08-06 06:33:35 on dist-test-slave-6zbq"
server_key: "92ec15286eb03bb790eb48a65176b51a"
server_key_iv: "54c111ae30fb539743e801216b9f9058"
server_key_version: "encryptionkey@0"
I20260806 06:33:36.768512 29185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:33:36.781234 29185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:33:36.783190 29185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:33:36.783304 29185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:33:36.783493 29185 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:33:36.783861 29263 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 06:33:36.785642 29185 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 06:33:36.785687 29185 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 06:33:36.785717 29185 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 06:33:36.786314 29185 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 06:33:36.786367 29185 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.001s
I20260806 06:33:36.786448 29263 tablet_bootstrap.cc:492] T abc465d6d67d41e09463dd7c9d9bc276 P 188fbd273bd64a288842883839b5326a: Bootstrap starting.
I20260806 06:33:36.792778 29185 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.65:42385
I20260806 06:33:36.792857 29415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.65:42385 every 8 connection(s)
I20260806 06:33:36.793143 29185 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
I20260806 06:33:36.796267 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 29185
I20260806 06:33:36.796528 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.66:38259
--local_ip_for_outbound_sockets=127.7.43.66
--tserver_master_addrs=127.7.43.126:40255
--webserver_port=45099
--webserver_interface=127.7.43.66
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab
--principal=kudu/127.7.43.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:35907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:33:36.797169 29263 log.cc:824] T abc465d6d67d41e09463dd7c9d9bc276 P 188fbd273bd64a288842883839b5326a: Log is configured to *not* fsync() on all Append() calls
I20260806 06:33:36.799829 29263 tablet_bootstrap.cc:492] T abc465d6d67d41e09463dd7c9d9bc276 P 188fbd273bd64a288842883839b5326a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:33:36.800136 29263 tablet_bootstrap.cc:492] T abc465d6d67d41e09463dd7c9d9bc276 P 188fbd273bd64a288842883839b5326a: Bootstrap complete.
I20260806 06:33:36.801290 29263 ts_tablet_manager.cc:1397] T abc465d6d67d41e09463dd7c9d9bc276 P 188fbd273bd64a288842883839b5326a: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.009s
I20260806 06:33:36.802655 29263 raft_consensus.cc:348] T abc465d6d67d41e09463dd7c9d9bc276 P 188fbd273bd64a288842883839b5326a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02ad740aae2d470cafcb96d12f589902" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 38259 } } peers { permanent_uuid: "194dbcf584f04a608c7f6a6a251d71e5" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 45915 } } peers { permanent_uuid: "188fbd273bd64a288842883839b5326a" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 42385 } }
I20260806 06:33:36.802984 29263 raft_consensus.cc:729] T abc465d6d67d41e09463dd7c9d9bc276 P 188fbd273bd64a288842883839b5326a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 188fbd273bd64a288842883839b5326a, State: Initialized, Role: FOLLOWER
I20260806 06:33:36.803122 29263 consensus_queue.cc:260] T abc465d6d67d41e09463dd7c9d9bc276 P 188fbd273bd64a288842883839b5326a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02ad740aae2d470cafcb96d12f589902" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 38259 } } peers { permanent_uuid: "194dbcf584f04a608c7f6a6a251d71e5" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 45915 } } peers { permanent_uuid: "188fbd273bd64a288842883839b5326a" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 42385 } }
I20260806 06:33:36.803380 29263 ts_tablet_manager.cc:1428] T abc465d6d67d41e09463dd7c9d9bc276 P 188fbd273bd64a288842883839b5326a: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
W20260806 06:33:36.856472 29418 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:33:36.793763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.65:46877)
0806 06:33:36.793955 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:36.793985 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 06:33:36.794431 (+ 446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:33:36.856288 (+ 61857us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":124,"threads_started":1}
W20260806 06:33:36.856702 29416 heartbeater.cc:644] Failed to heartbeat to 127.7.43.126:40255 (0 consecutive failures): Network error: Failed to ping master at 127.7.43.126:40255: Client connection negotiation failed: client connection to 127.7.43.126:40255: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 06:33:36.857020 29418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:36.856915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.65:40311)
0806 06:33:36.856961 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:36.857001 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260806 06:33:36.857374 29418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:36.857307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.65:49255)
0806 06:33:36.857341 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:36.857361 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260806 06:33:36.857475 29416 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 06:33:36.876398 29420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:33:36.876564 29420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:33:36.876586 29420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:33:36.877660 29420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:33:36.877692 29420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:33:36.877717 29420 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.66
I20260806 06:33:36.879106 29420 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:35907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab
--principal=kudu/127.7.43.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.66:38259
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.43.66
--webserver_port=45099
--tserver_master_addrs=127.7.43.126:40255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29420
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.66
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:33:36.879271 29420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:33:36.879432 29420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:33:36.883845 29431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:33:36.883912 29420 server_base.cc:1034] running on GCE node
W20260806 06:33:36.883858 29429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:33:36.883859 29427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:33:36.884200 29420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:33:36.884423 29420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:33:36.885574 29420 hybrid_clock.cc:648] HybridClock initialized: now 1785998016885559 us; error 26 us; skew 500 ppm
Aug 06 06:33:36 dist-test-slave-6zbq krb5kdc[27531](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998016, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:33:36.887866 29420 init.cc:377] Logged in from keytab as kudu/127.7.43.66@KRBTEST.COM (short username kudu)
I20260806 06:33:36.888229 29420 webserver.cc:460] Webserver started at http://127.7.43.66:45099/ using document root <none> and password file <none>
I20260806 06:33:36.888409 29420 fs_manager.cc:359] Metadata directory not provided
I20260806 06:33:36.888454 29420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:33:36.889605 29420 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:33:36.890153 29443 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:36.890300 29420 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:36.890398 29420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-1/wal
uuid: "02ad740aae2d470cafcb96d12f589902"
format_stamp: "Formatted at 2026-08-06 06:33:35 on dist-test-slave-6zbq"
server_key: "d3c084445a41564da8f73c40ecf31c75"
server_key_iv: "a3eb948d1b8ee2f36fe8d497a026ea36"
server_key_version: "encryptionkey@0"
I20260806 06:33:36.890671 29420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:33:36.899219 29420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:33:36.900760 29420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:33:36.900883 29420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:33:36.901062 29420 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:33:36.901414 29454 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 06:33:36.902683 29420 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 06:33:36.902722 29420 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20260806 06:33:36.902750 29420 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 06:33:36.903332 29420 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 06:33:36.903365 29420 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 06:33:36.903412 29454 tablet_bootstrap.cc:492] T abc465d6d67d41e09463dd7c9d9bc276 P 02ad740aae2d470cafcb96d12f589902: Bootstrap starting.
I20260806 06:33:36.909792 29420 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.66:38259
I20260806 06:33:36.909875 29619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.66:38259 every 8 connection(s)
I20260806 06:33:36.910147 29420 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
I20260806 06:33:36.911154 29622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:36.910719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.66:50415)
0806 06:33:36.910888 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:36.910988 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":73,"threads_started":1}
W20260806 06:33:36.911391 29620 heartbeater.cc:644] Failed to heartbeat to 127.7.43.126:40255 (0 consecutive failures): Network error: Failed to ping master at 127.7.43.126:40255: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
I20260806 06:33:36.911670 29622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:36.911564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.66:44649)
0806 06:33:36.911611 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:36.911644 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 06:33:36.911769 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 29420
I20260806 06:33:36.912036 29622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:36.911871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.66:55753)
0806 06:33:36.911987 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:36.912011 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":74,"threads_started":1}
I20260806 06:33:36.912048 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.67:45915
--local_ip_for_outbound_sockets=127.7.43.67
--tserver_master_addrs=127.7.43.126:40255
--webserver_port=39839
--webserver_interface=127.7.43.67
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab
--principal=kudu/127.7.43.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:35907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:33:36.912128 29620 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 06:33:36.912331 29454 log.cc:824] T abc465d6d67d41e09463dd7c9d9bc276 P 02ad740aae2d470cafcb96d12f589902: Log is configured to *not* fsync() on all Append() calls
I20260806 06:33:36.914458 29454 tablet_bootstrap.cc:492] T abc465d6d67d41e09463dd7c9d9bc276 P 02ad740aae2d470cafcb96d12f589902: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:33:36.914740 29454 tablet_bootstrap.cc:492] T abc465d6d67d41e09463dd7c9d9bc276 P 02ad740aae2d470cafcb96d12f589902: Bootstrap complete.
I20260806 06:33:36.915489 29454 ts_tablet_manager.cc:1397] T abc465d6d67d41e09463dd7c9d9bc276 P 02ad740aae2d470cafcb96d12f589902: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.003s
I20260806 06:33:36.916581 29454 raft_consensus.cc:348] T abc465d6d67d41e09463dd7c9d9bc276 P 02ad740aae2d470cafcb96d12f589902 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02ad740aae2d470cafcb96d12f589902" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 38259 } } peers { permanent_uuid: "194dbcf584f04a608c7f6a6a251d71e5" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 45915 } } peers { permanent_uuid: "188fbd273bd64a288842883839b5326a" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 42385 } }
I20260806 06:33:36.916810 29454 raft_consensus.cc:729] T abc465d6d67d41e09463dd7c9d9bc276 P 02ad740aae2d470cafcb96d12f589902 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02ad740aae2d470cafcb96d12f589902, State: Initialized, Role: FOLLOWER
I20260806 06:33:36.916908 29454 consensus_queue.cc:260] T abc465d6d67d41e09463dd7c9d9bc276 P 02ad740aae2d470cafcb96d12f589902 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02ad740aae2d470cafcb96d12f589902" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 38259 } } peers { permanent_uuid: "194dbcf584f04a608c7f6a6a251d71e5" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 45915 } } peers { permanent_uuid: "188fbd273bd64a288842883839b5326a" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 42385 } }
I20260806 06:33:36.917137 29454 ts_tablet_manager.cc:1428] T abc465d6d67d41e09463dd7c9d9bc276 P 02ad740aae2d470cafcb96d12f589902: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:33:36.917320 29622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:36.917243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.66:48197)
0806 06:33:36.917272 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:36.917300 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260806 06:33:36.917562 29622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:36.917484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.66:45929)
0806 06:33:36.917524 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:36.917546 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260806 06:33:36.992362 29625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:33:36.992537 29625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:33:36.992558 29625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:33:36.993671 29625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:33:36.993718 29625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:33:36.993744 29625 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.67
I20260806 06:33:36.995158 29625 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:35907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab
--principal=kudu/127.7.43.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.67:45915
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.43.67
--webserver_port=39839
--tserver_master_addrs=127.7.43.126:40255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29625
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.67
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:33:36.995319 29625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:33:36.995477 29625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:33:36.999961 29634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:33:37.000082 29625 server_base.cc:1034] running on GCE node
W20260806 06:33:37.000025 29636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:33:37.000048 29638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:33:37.000384 29625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:33:37.000626 29625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:33:37.001765 29625 hybrid_clock.cc:648] HybridClock initialized: now 1785998017001751 us; error 28 us; skew 500 ppm
Aug 06 06:33:37 dist-test-slave-6zbq krb5kdc[27531](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998017, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:33:37.004066 29625 init.cc:377] Logged in from keytab as kudu/127.7.43.67@KRBTEST.COM (short username kudu)
I20260806 06:33:37.004463 29625 webserver.cc:460] Webserver started at http://127.7.43.67:39839/ using document root <none> and password file <none>
I20260806 06:33:37.004624 29625 fs_manager.cc:359] Metadata directory not provided
I20260806 06:33:37.004662 29625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:33:37.005791 29625 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:33:37.006279 29647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:37.006443 29625 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:37.006510 29625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-2/wal
uuid: "194dbcf584f04a608c7f6a6a251d71e5"
format_stamp: "Formatted at 2026-08-06 06:33:35 on dist-test-slave-6zbq"
server_key: "4d996aa994a70e5b70ed78697afbc27c"
server_key_iv: "0e2e1d01e6bd2bb93a2cedfca6252a31"
server_key_version: "encryptionkey@0"
I20260806 06:33:37.006769 29625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:33:37.040549 29625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:33:37.042076 29625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:33:37.042183 29625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:33:37.042368 29625 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:33:37.042615 29625 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:33:37.042645 29625 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:37.042665 29625 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:33:37.042680 29625 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:37.048316 29625 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.67:45915
I20260806 06:33:37.048396 29817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.67:45915 every 8 connection(s)
I20260806 06:33:37.049098 29625 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
I20260806 06:33:37.049846 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 29625
I20260806 06:33:37.050099 29820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:37.049666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.67:50557)
0806 06:33:37.049848 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:37.049969 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":93,"threads_started":1}
W20260806 06:33:37.050464 29818 heartbeater.cc:644] Failed to heartbeat to 127.7.43.126:40255 (0 consecutive failures): Network error: Failed to ping master at 127.7.43.126:40255: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
I20260806 06:33:37.050717 29820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:37.050624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.67:34791)
0806 06:33:37.050662 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:37.050686 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260806 06:33:37.050949 29820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:37.050877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.67:35261)
0806 06:33:37.050910 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:37.050930 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260806 06:33:37.051038 29818 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 06:33:37.857988 29823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:37.857778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.65:39825)
0806 06:33:37.857897 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:37.857931 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":56,"threads_started":1}
I20260806 06:33:37.918044 29825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:37.917859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.66:41045)
0806 06:33:37.917969 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:37.918005 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":53,"threads_started":1}
I20260806 06:33:38.051445 29828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:38.051270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.67:46589)
0806 06:33:38.051368 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:38.051394 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":52,"threads_started":1}
I20260806 06:33:38.407737 29829 raft_consensus.cc:482] T abc465d6d67d41e09463dd7c9d9bc276 P 188fbd273bd64a288842883839b5326a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:33:38.407852 29829 raft_consensus.cc:504] T abc465d6d67d41e09463dd7c9d9bc276 P 188fbd273bd64a288842883839b5326a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02ad740aae2d470cafcb96d12f589902" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 38259 } } peers { permanent_uuid: "194dbcf584f04a608c7f6a6a251d71e5" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 45915 } } peers { permanent_uuid: "188fbd273bd64a288842883839b5326a" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 42385 } }
I20260806 06:33:38.408177 29829 leader_election.cc:288] T abc465d6d67d41e09463dd7c9d9bc276 P 188fbd273bd64a288842883839b5326a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 02ad740aae2d470cafcb96d12f589902 (127.7.43.66:38259), 194dbcf584f04a608c7f6a6a251d71e5 (127.7.43.67:45915)
Aug 06 06:33:38 dist-test-slave-6zbq krb5kdc[27531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998016, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.65@KRBTEST.COM for kudu/127.7.43.67@KRBTEST.COM
Aug 06 06:33:38 dist-test-slave-6zbq krb5kdc[27531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998016, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.65@KRBTEST.COM for kudu/127.7.43.66@KRBTEST.COM
I20260806 06:33:38.420857 29832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:38.408412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:42399 (local address 127.7.43.66:38259)
0806 06:33:38.408582 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:38.408595 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:38.408620 (+ 25us) server_negotiation.cc:407] Connection header received
0806 06:33:38.408735 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:38.408736 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:38.408777 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:38.408883 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:33:38.409757 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:38.410671 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:38.411725 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:38.412108 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:38.412207 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:38.417823 (+ 5616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:33:38.417826 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:33:38.417826 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:33:38.417836 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:33:38.419508 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:38.419820 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:38.419823 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:38.419823 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:38.419899 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:38.420059 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:38.420060 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:38.420060 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:38.420323 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:33:38.420392 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:38.420631 (+ 239us) server_negotiation.cc:299] Negotiation successful
0806 06:33:38.420760 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":52,"threads_started":1}
I20260806 06:33:38.420853 29831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:38.408333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.67:45915 (local address 127.7.43.65:59613)
0806 06:33:38.408522 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:38.408531 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:33:38.408571 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:33:38.409431 (+ 860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:33:38.409432 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:33:38.409612 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:33:38.409684 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:38.409687 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:38.410757 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:38.410758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:38.411643 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:38.411645 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:38.412190 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:38.412191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:38.412306 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:38.415920 (+ 3614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:33:38.415930 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:33:38.417371 (+ 1441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:33:38.419361 (+ 1990us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:33:38.419368 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:33:38.419370 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:33:38.419587 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:33:38.419797 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:33:38.419797 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:33:38.419798 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:33:38.419904 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:33:38.420317 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:33:38.420322 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:33:38.420459 (+ 137us) client_negotiation.cc:769] Sending connection context
0806 06:33:38.420573 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 06:33:38.420744 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"mutex_wait_us":69,"thread_start_us":95,"threads_started":1}
I20260806 06:33:38.420871 29833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:38.408523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:59613 (local address 127.7.43.67:45915)
0806 06:33:38.408663 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:38.408672 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:38.408700 (+ 28us) server_negotiation.cc:407] Connection header received
0806 06:33:38.409250 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:38.409253 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:38.409301 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:38.409435 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:33:38.409791 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:38.410665 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:38.411719 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:38.412108 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:38.412206 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:38.417469 (+ 5263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:33:38.417474 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:33:38.417474 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:33:38.417484 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:33:38.419286 (+ 1802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:38.419657 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:38.419659 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:38.419659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:38.419739 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:38.419965 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:38.419966 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:38.419966 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:38.420252 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:33:38.420334 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:38.420636 (+ 302us) server_negotiation.cc:299] Negotiation successful
0806 06:33:38.420760 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":49,"threads_started":1}
I20260806 06:33:38.420914 29830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:38.408333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.66:38259 (local address 127.7.43.65:42399)
0806 06:33:38.408443 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:38.408453 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:33:38.408508 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:33:38.408944 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:33:38.408947 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:33:38.409329 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:33:38.409629 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:38.409637 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:38.410773 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:38.410774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:38.411649 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:38.411651 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:38.412194 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:38.412195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:38.412306 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:38.415927 (+ 3621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:33:38.415932 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:33:38.417733 (+ 1801us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:33:38.419597 (+ 1864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:33:38.419598 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:33:38.419598 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:33:38.419751 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:33:38.419955 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:33:38.419955 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:33:38.419955 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:33:38.420008 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:33:38.420378 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:33:38.420379 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:33:38.420459 (+ 80us) client_negotiation.cc:769] Sending connection context
0806 06:33:38.420573 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 06:33:38.420808 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"mutex_wait_us":118,"thread_start_us":54,"threads_started":1}
W20260806 06:33:38.421768 29258 leader_election.cc:341] T abc465d6d67d41e09463dd7c9d9bc276 P 188fbd273bd64a288842883839b5326a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 194dbcf584f04a608c7f6a6a251d71e5 (127.7.43.67:45915): Not found: Tablet not found: abc465d6d67d41e09463dd7c9d9bc276
I20260806 06:33:38.422060 29554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abc465d6d67d41e09463dd7c9d9bc276" candidate_uuid: "188fbd273bd64a288842883839b5326a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "02ad740aae2d470cafcb96d12f589902" is_pre_election: true
I20260806 06:33:38.422185 29554 raft_consensus.cc:2454] T abc465d6d67d41e09463dd7c9d9bc276 P 02ad740aae2d470cafcb96d12f589902 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 188fbd273bd64a288842883839b5326a in term 1.
I20260806 06:33:38.422423 29256 leader_election.cc:302] T abc465d6d67d41e09463dd7c9d9bc276 P 188fbd273bd64a288842883839b5326a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 02ad740aae2d470cafcb96d12f589902, 188fbd273bd64a288842883839b5326a; no voters: 194dbcf584f04a608c7f6a6a251d71e5
I20260806 06:33:38.422536 29829 raft_consensus.cc:2781] T abc465d6d67d41e09463dd7c9d9bc276 P 188fbd273bd64a288842883839b5326a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 06:33:38.422600 29829 raft_consensus.cc:482] T abc465d6d67d41e09463dd7c9d9bc276 P 188fbd273bd64a288842883839b5326a [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:33:38.422623 29829 raft_consensus.cc:3037] T abc465d6d67d41e09463dd7c9d9bc276 P 188fbd273bd64a288842883839b5326a [term 1 FOLLOWER]: Advancing to term 2
I20260806 06:33:38.423691 29829 raft_consensus.cc:504] T abc465d6d67d41e09463dd7c9d9bc276 P 188fbd273bd64a288842883839b5326a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02ad740aae2d470cafcb96d12f589902" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 38259 } } peers { permanent_uuid: "194dbcf584f04a608c7f6a6a251d71e5" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 45915 } } peers { permanent_uuid: "188fbd273bd64a288842883839b5326a" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 42385 } }
I20260806 06:33:38.423834 29829 leader_election.cc:288] T abc465d6d67d41e09463dd7c9d9bc276 P 188fbd273bd64a288842883839b5326a [CANDIDATE]: Term 2 election: Requested vote from peers 02ad740aae2d470cafcb96d12f589902 (127.7.43.66:38259), 194dbcf584f04a608c7f6a6a251d71e5 (127.7.43.67:45915)
I20260806 06:33:38.424011 29554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abc465d6d67d41e09463dd7c9d9bc276" candidate_uuid: "188fbd273bd64a288842883839b5326a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "02ad740aae2d470cafcb96d12f589902"
I20260806 06:33:38.424094 29554 raft_consensus.cc:3037] T abc465d6d67d41e09463dd7c9d9bc276 P 02ad740aae2d470cafcb96d12f589902 [term 1 FOLLOWER]: Advancing to term 2
W20260806 06:33:38.424104 29258 leader_election.cc:341] T abc465d6d67d41e09463dd7c9d9bc276 P 188fbd273bd64a288842883839b5326a [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 194dbcf584f04a608c7f6a6a251d71e5 (127.7.43.67:45915): Not found: Tablet not found: abc465d6d67d41e09463dd7c9d9bc276
I20260806 06:33:38.425127 29554 raft_consensus.cc:2454] T abc465d6d67d41e09463dd7c9d9bc276 P 02ad740aae2d470cafcb96d12f589902 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 188fbd273bd64a288842883839b5326a in term 2.
I20260806 06:33:38.425313 29256 leader_election.cc:302] T abc465d6d67d41e09463dd7c9d9bc276 P 188fbd273bd64a288842883839b5326a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 02ad740aae2d470cafcb96d12f589902, 188fbd273bd64a288842883839b5326a; no voters: 194dbcf584f04a608c7f6a6a251d71e5
I20260806 06:33:38.425400 29829 raft_consensus.cc:2781] T abc465d6d67d41e09463dd7c9d9bc276 P 188fbd273bd64a288842883839b5326a [term 2 FOLLOWER]: Leader election won for term 2
I20260806 06:33:38.425508 29829 raft_consensus.cc:686] T abc465d6d67d41e09463dd7c9d9bc276 P 188fbd273bd64a288842883839b5326a [term 2 LEADER]: Becoming Leader. State: Replica: 188fbd273bd64a288842883839b5326a, State: Running, Role: LEADER
I20260806 06:33:38.425588 29829 consensus_queue.cc:237] T abc465d6d67d41e09463dd7c9d9bc276 P 188fbd273bd64a288842883839b5326a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02ad740aae2d470cafcb96d12f589902" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 38259 } } peers { permanent_uuid: "194dbcf584f04a608c7f6a6a251d71e5" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 45915 } } peers { permanent_uuid: "188fbd273bd64a288842883839b5326a" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 42385 } }
I20260806 06:33:38.425863 29831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:38.425684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.65:36773)
0806 06:33:38.425804 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:38.425836 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260806 06:33:38.811303 29554 raft_consensus.cc:1261] T abc465d6d67d41e09463dd7c9d9bc276 P 02ad740aae2d470cafcb96d12f589902 [term 2 FOLLOWER]: Refusing update from remote peer 188fbd273bd64a288842883839b5326a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260806 06:33:38.811623 29829 consensus_queue.cc:1035] T abc465d6d67d41e09463dd7c9d9bc276 P 188fbd273bd64a288842883839b5326a [LEADER]: Connected to new peer: Peer: permanent_uuid: "02ad740aae2d470cafcb96d12f589902" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 38259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260806 06:33:38.812448 29846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:38.812297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.66:34273)
0806 06:33:38.812378 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:38.812412 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71,"thread_start_us":44,"threads_started":1}
I20260806 06:33:38.813251 29831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:38.813157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.65:40611)
0806 06:33:38.813195 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:38.813222 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260806 06:33:38.813429 29258 consensus_peers.cc:487] T abc465d6d67d41e09463dd7c9d9bc276 P 188fbd273bd64a288842883839b5326a -> Peer 194dbcf584f04a608c7f6a6a251d71e5 (127.7.43.67:45915): Couldn't send request to peer 194dbcf584f04a608c7f6a6a251d71e5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: abc465d6d67d41e09463dd7c9d9bc276. This is attempt 1: this message will repeat every 5th retry.
I20260806 06:33:38.813489 29846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:38.813394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.66:57559)
0806 06:33:38.813433 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:38.813462 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260806 06:33:39.051967 29850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:39.051780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.67:50847)
0806 06:33:39.051899 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:39.051930 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":57,"threads_started":1}
I20260806 06:33:39.394837 29854 ts_tablet_manager.cc:927] T abc465d6d67d41e09463dd7c9d9bc276 P 194dbcf584f04a608c7f6a6a251d71e5: Initiating tablet copy from peer 188fbd273bd64a288842883839b5326a (127.7.43.65:42385)
I20260806 06:33:39.395041 29854 tablet_copy_client.cc:316] T abc465d6d67d41e09463dd7c9d9bc276 P 194dbcf584f04a608c7f6a6a251d71e5: tablet copy: Beginning tablet copy session from remote peer at address 127.7.43.65:42385
Aug 06 06:33:39 dist-test-slave-6zbq krb5kdc[27531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998017, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.67@KRBTEST.COM for kudu/127.7.43.65@KRBTEST.COM
I20260806 06:33:39.402119 29856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:39.395369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:46483 (local address 127.7.43.65:42385)
0806 06:33:39.395495 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:39.395500 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:39.395509 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:33:39.395533 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:39.395535 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:39.395547 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:39.395626 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:33:39.396011 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:39.396701 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:39.397421 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:39.397702 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:39.397750 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:39.399819 (+ 2069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:33:39.399827 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:33:39.399828 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:33:39.399834 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:33:39.401169 (+ 1335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:39.401510 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:39.401512 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:39.401512 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:39.401561 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:39.401694 (+ 133us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:39.401694 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:39.401695 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:39.401819 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:33:39.401875 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:39.401993 (+ 118us) server_negotiation.cc:299] Negotiation successful
0806 06:33:39.402051 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":68,"threads_started":1}
I20260806 06:33:39.402125 29850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:39.395269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:42385 (local address 127.7.43.67:46483)
0806 06:33:39.395321 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:39.395342 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 06:33:39.395418 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:33:39.395626 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:33:39.395629 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:33:39.395891 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:33:39.395951 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:39.395953 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:39.396781 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:39.396782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:39.397359 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:39.397361 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:39.397752 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:39.397752 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:39.397802 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:39.398252 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:33:39.398256 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:33:39.399732 (+ 1476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:33:39.401251 (+ 1519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:33:39.401258 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:33:39.401260 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:33:39.401448 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:33:39.401604 (+ 156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:33:39.401605 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:33:39.401605 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:33:39.401655 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:33:39.401862 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:33:39.401865 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:33:39.401906 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 06:33:39.401965 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 06:33:39.402046 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 06:33:39.402561 29382 tablet_copy_service.cc:140] P 188fbd273bd64a288842883839b5326a: Received BeginTabletCopySession request for tablet abc465d6d67d41e09463dd7c9d9bc276 from peer 194dbcf584f04a608c7f6a6a251d71e5 ({username='kudu', principal='kudu/127.7.43.67@KRBTEST.COM'} at 127.7.43.67:46483)
I20260806 06:33:39.402623 29382 tablet_copy_service.cc:161] P 188fbd273bd64a288842883839b5326a: Beginning new tablet copy session on tablet abc465d6d67d41e09463dd7c9d9bc276 from peer 194dbcf584f04a608c7f6a6a251d71e5 at {username='kudu', principal='kudu/127.7.43.67@KRBTEST.COM'} at 127.7.43.67:46483: session id = 194dbcf584f04a608c7f6a6a251d71e5-abc465d6d67d41e09463dd7c9d9bc276
I20260806 06:33:39.403064 29382 tablet_copy_source_session.cc:205] T abc465d6d67d41e09463dd7c9d9bc276 P 188fbd273bd64a288842883839b5326a: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 06:33:39.403975 29854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abc465d6d67d41e09463dd7c9d9bc276. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:33:39.406490 29854 tablet_copy_client.cc:791] T abc465d6d67d41e09463dd7c9d9bc276 P 194dbcf584f04a608c7f6a6a251d71e5: tablet copy: Starting download of 0 data blocks...
I20260806 06:33:39.406623 29854 tablet_copy_client.cc:655] T abc465d6d67d41e09463dd7c9d9bc276 P 194dbcf584f04a608c7f6a6a251d71e5: tablet copy: Starting download of 1 WAL segments...
I20260806 06:33:39.407357 29854 tablet_copy_client.cc:531] T abc465d6d67d41e09463dd7c9d9bc276 P 194dbcf584f04a608c7f6a6a251d71e5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 06:33:39.407981 29854 tablet_bootstrap.cc:492] T abc465d6d67d41e09463dd7c9d9bc276 P 194dbcf584f04a608c7f6a6a251d71e5: Bootstrap starting.
I20260806 06:33:39.409312 29854 log.cc:824] T abc465d6d67d41e09463dd7c9d9bc276 P 194dbcf584f04a608c7f6a6a251d71e5: Log is configured to *not* fsync() on all Append() calls
I20260806 06:33:39.411731 29854 tablet_bootstrap.cc:492] T abc465d6d67d41e09463dd7c9d9bc276 P 194dbcf584f04a608c7f6a6a251d71e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:33:39.411923 29854 tablet_bootstrap.cc:492] T abc465d6d67d41e09463dd7c9d9bc276 P 194dbcf584f04a608c7f6a6a251d71e5: Bootstrap complete.
I20260806 06:33:39.412032 29854 ts_tablet_manager.cc:1397] T abc465d6d67d41e09463dd7c9d9bc276 P 194dbcf584f04a608c7f6a6a251d71e5: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.002s
I20260806 06:33:39.412992 29854 raft_consensus.cc:348] T abc465d6d67d41e09463dd7c9d9bc276 P 194dbcf584f04a608c7f6a6a251d71e5 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02ad740aae2d470cafcb96d12f589902" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 38259 } } peers { permanent_uuid: "194dbcf584f04a608c7f6a6a251d71e5" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 45915 } } peers { permanent_uuid: "188fbd273bd64a288842883839b5326a" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 42385 } }
I20260806 06:33:39.413134 29854 raft_consensus.cc:729] T abc465d6d67d41e09463dd7c9d9bc276 P 194dbcf584f04a608c7f6a6a251d71e5 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 194dbcf584f04a608c7f6a6a251d71e5, State: Initialized, Role: FOLLOWER
I20260806 06:33:39.413245 29854 consensus_queue.cc:260] T abc465d6d67d41e09463dd7c9d9bc276 P 194dbcf584f04a608c7f6a6a251d71e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02ad740aae2d470cafcb96d12f589902" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 38259 } } peers { permanent_uuid: "194dbcf584f04a608c7f6a6a251d71e5" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 45915 } } peers { permanent_uuid: "188fbd273bd64a288842883839b5326a" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 42385 } }
I20260806 06:33:39.413483 29854 ts_tablet_manager.cc:1428] T abc465d6d67d41e09463dd7c9d9bc276 P 194dbcf584f04a608c7f6a6a251d71e5: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20260806 06:33:39.413723 29850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:39.413639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.67:39121)
0806 06:33:39.413668 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:39.413696 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
I20260806 06:33:39.413714 29382 tablet_copy_service.cc:342] P 188fbd273bd64a288842883839b5326a: Request end of tablet copy session 194dbcf584f04a608c7f6a6a251d71e5-abc465d6d67d41e09463dd7c9d9bc276 received from {username='kudu', principal='kudu/127.7.43.67@KRBTEST.COM'} at 127.7.43.67:46483
I20260806 06:33:39.413757 29382 tablet_copy_service.cc:433] P 188fbd273bd64a288842883839b5326a: ending tablet copy session 194dbcf584f04a608c7f6a6a251d71e5-abc465d6d67d41e09463dd7c9d9bc276 on tablet abc465d6d67d41e09463dd7c9d9bc276 with peer 194dbcf584f04a608c7f6a6a251d71e5
I20260806 06:33:39.414031 29850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:39.413938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.67:55721)
0806 06:33:39.413980 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:39.414006 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260806 06:33:39.813827 29865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:39.813618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.65:45875)
0806 06:33:39.813753 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:39.813785 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":66,"threads_started":1}
I20260806 06:33:39.813964 29866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:39.813790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.66:41089)
0806 06:33:39.813884 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:39.813918 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78,"thread_start_us":45,"threads_started":1}
I20260806 06:33:39.949095 29865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:39.948987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.65:49467)
0806 06:33:39.949033 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:39.949058 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260806 06:33:39.949124 29756 raft_consensus.cc:1203] T abc465d6d67d41e09463dd7c9d9bc276 P 194dbcf584f04a608c7f6a6a251d71e5 [term 2 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[2.3-2.3] Dedup: 2.3->[]
I20260806 06:33:39.949627 29873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:39.949489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.67:50309)
0806 06:33:39.949569 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:39.949598 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68,"thread_start_us":43,"threads_started":1}
I20260806 06:33:40.814500 29874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:40.814303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.66:43561)
0806 06:33:40.814427 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:40.814459 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":81,"threads_started":1}
I20260806 06:33:40.949601 29875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:40.949423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.65:44159)
0806 06:33:40.949528 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:40.949560 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":59,"threads_started":1}
I20260806 06:33:40.950119 29878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:40.949944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.67:36881)
0806 06:33:40.950054 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:40.950081 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":48,"threads_started":1}
I20260806 06:33:41.815022 29879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:41.814847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.66:42387)
0806 06:33:41.814952 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:41.814982 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":62,"threads_started":1}
I20260806 06:33:41.950194 29880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:41.950005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.65:43647)
0806 06:33:41.950120 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:41.950154 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":59,"threads_started":1}
I20260806 06:33:41.950567 29880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:41.950483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.65:35769)
0806 06:33:41.950524 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:41.950547 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260806 06:33:41.950690 29881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:41.950501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.67:48753)
0806 06:33:41.950611 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:41.950644 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":57,"threads_started":1}
I20260806 06:33:42.815618 29883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:42.815418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.66:51967)
0806 06:33:42.815536 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:42.815569 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":68,"threads_started":1}
I20260806 06:33:42.951058 29884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:42.950880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.65:54615)
0806 06:33:42.950986 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:42.951015 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":56,"threads_started":1}
I20260806 06:33:42.951241 29885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:42.951058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.67:33207)
0806 06:33:42.951166 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:42.951195 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":52,"threads_started":1}
I20260806 06:33:43.816159 29886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:43.815961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.66:56761)
0806 06:33:43.816082 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:43.816114 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":69,"threads_started":1}
I20260806 06:33:43.951702 29889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:43.951478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.65:37087)
0806 06:33:43.951612 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:43.951647 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":69,"threads_started":1}
I20260806 06:33:43.951761 29890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:43.951596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.67:39961)
0806 06:33:43.951692 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:43.951720 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81,"thread_start_us":44,"threads_started":1}
I20260806 06:33:44.816674 29891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:44.816501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.66:57711)
0806 06:33:44.816604 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:44.816633 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":60,"threads_started":1}
I20260806 06:33:44.952262 29893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:44.952080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.65:39351)
0806 06:33:44.952190 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:44.952219 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":57,"threads_started":1}
I20260806 06:33:44.952368 29896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:44.952152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.67:55731)
0806 06:33:44.952291 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:44.952323 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":69,"threads_started":1}
I20260806 06:33:45.817262 29898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:45.817052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.66:60307)
0806 06:33:45.817176 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:45.817210 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":72,"threads_started":1}
I20260806 06:33:45.952842 29899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:45.952641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.65:45333)
0806 06:33:45.952758 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:45.952788 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":71,"threads_started":1}
I20260806 06:33:45.952886 29901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:45.952712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.67:49219)
0806 06:33:45.952818 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:45.952847 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":53,"threads_started":1}
I20260806 06:33:46.817826 29902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:46.817617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.66:54025)
0806 06:33:46.817747 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:46.817783 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":61,"threads_started":1}
I20260806 06:33:46.953456 29903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:46.953257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.67:59227)
0806 06:33:46.953383 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:46.953414 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":58,"threads_started":1}
I20260806 06:33:46.953456 29904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:46.953273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.65:51835)
0806 06:33:46.953384 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:46.953413 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":58,"threads_started":1}
I20260806 06:33:47.818390 29905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:47.818191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.66:60553)
0806 06:33:47.818300 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:47.818334 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":57,"threads_started":1}
I20260806 06:33:47.954032 29907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:47.953839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.65:36427)
0806 06:33:47.953957 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:47.953987 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":59,"threads_started":1}
I20260806 06:33:47.954036 29908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:47.953857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.67:50039)
0806 06:33:47.953964 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:47.953995 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":56,"threads_started":1}
I20260806 06:33:48.818933 29909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:48.818748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.66:49265)
0806 06:33:48.818857 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:48.818888 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":60,"threads_started":1}
I20260806 06:33:48.954654 29910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:48.954459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.67:50737)
0806 06:33:48.954576 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:48.954609 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":66,"threads_started":1}
I20260806 06:33:48.954663 29912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:48.954459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.65:52881)
0806 06:33:48.954600 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:48.954625 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":69,"threads_started":1}
I20260806 06:33:49.819525 29914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:49.819329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.66:60547)
0806 06:33:49.819446 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:49.819479 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":60,"threads_started":1}
I20260806 06:33:49.955274 29917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:49.955055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.65:34635)
0806 06:33:49.955194 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:49.955233 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":71,"threads_started":1}
I20260806 06:33:49.955272 29915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:49.955056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.67:33515)
0806 06:33:49.955198 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:49.955231 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":58,"threads_started":1}
I20260806 06:33:50.820108 29919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:50.819891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.66:54465)
0806 06:33:50.820028 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:50.820062 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":72,"threads_started":1}
I20260806 06:33:50.955849 29921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:50.955637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.67:51443)
0806 06:33:50.955775 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:50.955807 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":75,"threads_started":1}
I20260806 06:33:50.955849 29920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:50.955651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.65:33667)
0806 06:33:50.955773 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:50.955805 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":57,"threads_started":1}
I20260806 06:33:51.820663 29923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:51.820477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.66:33755)
0806 06:33:51.820590 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:51.820623 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":58,"threads_started":1}
I20260806 06:33:51.956411 29928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:51.956242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.65:36391)
0806 06:33:51.956344 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:51.956372 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":54,"threads_started":1}
I20260806 06:33:51.956408 29926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:51.956230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40255 (local address 127.7.43.67:50419)
0806 06:33:51.956339 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:51.956365 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:40255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":56,"threads_started":1}
I20260806 06:33:52.056721 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 29185
I20260806 06:33:52.066180 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 29420
I20260806 06:33:52.076436 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 29625
2026-08-06T06:33:52Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18180 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:33:52 dist-test-slave-6zbq krb5kdc[29932](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:33:52 dist-test-slave-6zbq krb5kdc[29932](info): set up 2 sockets
Aug 06 06:33:52 dist-test-slave-6zbq krb5kdc[29932](info): commencing operation
krb5kdc: starting...
W20260806 06:33:53.175731 7341 mini_kdc.cc:121] Time spent starting KDC: real 1.075s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Aug 06 06:33:53 dist-test-slave-6zbq krb5kdc[29932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998033, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:33:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:33:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.43.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
I20260806 06:33:53.281927 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:33109
--webserver_interface=127.7.43.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:40943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.43.126:33109
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:33:53.359280 29948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:33:53.359457 29948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:33:53.359476 29948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:33:53.360559 29948 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 06:33:53.360596 29948 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 06:33:53.360607 29948 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 06:33:53.360617 29948 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:33:53.360637 29948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:33:53.360662 29948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:33:53.360684 29948 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:33:53.360707 29948 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:33:53.362365 29948 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:40943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.43.126:33109
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:33109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.43.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29948
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:33:53.362598 29948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:33:53.362800 29948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:33:53.367388 29953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:33:53.367401 29954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:33:53.367403 29957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:33:53.367491 29948 server_base.cc:1034] running on GCE node
I20260806 06:33:53.367831 29948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:33:53.368063 29948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:33:53.369197 29948 hybrid_clock.cc:648] HybridClock initialized: now 1785998033369180 us; error 34 us; skew 500 ppm
Aug 06 06:33:53 dist-test-slave-6zbq krb5kdc[29932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998033, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:33:53.371417 29948 init.cc:377] Logged in from keytab as kudu/127.7.43.126@KRBTEST.COM (short username kudu)
I20260806 06:33:53.371814 29948 webserver.cc:460] Webserver started at http://127.7.43.126:42165/ using document root <none> and password file <none>
I20260806 06:33:53.371995 29948 fs_manager.cc:359] Metadata directory not provided
I20260806 06:33:53.372040 29948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:33:53.372153 29948 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:33:53.373103 29948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/master-0/data/instance:
uuid: "8e48a8a8158c4faeaf4beb84cddceaa7"
format_stamp: "Formatted at 2026-08-06 06:33:53 on dist-test-slave-6zbq"
server_key: "d8709f3cd43533890902d511c9bc7330"
server_key_iv: "57e178add804559622f849671220d387"
server_key_version: "encryptionkey@0"
I20260806 06:33:53.373433 29948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/master-0/wal/instance:
uuid: "8e48a8a8158c4faeaf4beb84cddceaa7"
format_stamp: "Formatted at 2026-08-06 06:33:53 on dist-test-slave-6zbq"
server_key: "d8709f3cd43533890902d511c9bc7330"
server_key_iv: "57e178add804559622f849671220d387"
server_key_version: "encryptionkey@0"
I20260806 06:33:53.374647 29948 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:33:53.375531 29966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:53.375689 29948 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:33:53.375792 29948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "8e48a8a8158c4faeaf4beb84cddceaa7"
format_stamp: "Formatted at 2026-08-06 06:33:53 on dist-test-slave-6zbq"
server_key: "d8709f3cd43533890902d511c9bc7330"
server_key_iv: "57e178add804559622f849671220d387"
server_key_version: "encryptionkey@0"
I20260806 06:33:53.375857 29948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:33:53.383009 29948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:33:53.384511 29948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:33:53.384629 29948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:33:53.388252 29948 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.126:33109
I20260806 06:33:53.388280 30050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.126:33109 every 8 connection(s)
I20260806 06:33:53.388566 29948 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:33:53.389326 30051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:33:53.391373 30051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7: Bootstrap starting.
I20260806 06:33:53.391928 30051 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7: Neither blocks nor log segments found. Creating new log.
I20260806 06:33:53.392160 30051 log.cc:824] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7: Log is configured to *not* fsync() on all Append() calls
I20260806 06:33:53.392722 30051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7: No bootstrap required, opened a new log
I20260806 06:33:53.394165 30051 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e48a8a8158c4faeaf4beb84cddceaa7" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 33109 } }
I20260806 06:33:53.394285 30051 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:33:53.394320 30051 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e48a8a8158c4faeaf4beb84cddceaa7, State: Initialized, Role: FOLLOWER
I20260806 06:33:53.394446 30051 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e48a8a8158c4faeaf4beb84cddceaa7" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 33109 } }
I20260806 06:33:53.394505 30051 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:33:53.394524 30051 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:33:53.394553 30051 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:33:53.394991 30051 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e48a8a8158c4faeaf4beb84cddceaa7" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 33109 } }
I20260806 06:33:53.395069 30051 leader_election.cc:302] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e48a8a8158c4faeaf4beb84cddceaa7; no voters:
I20260806 06:33:53.395187 30051 leader_election.cc:288] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:33:53.395256 30058 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:33:53.395397 30051 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:33:53.395426 30058 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7 [term 1 LEADER]: Becoming Leader. State: Replica: 8e48a8a8158c4faeaf4beb84cddceaa7, State: Running, Role: LEADER
I20260806 06:33:53.395556 30058 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e48a8a8158c4faeaf4beb84cddceaa7" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 33109 } }
I20260806 06:33:53.395685 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 29948
I20260806 06:33:53.395818 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/master-0/wal/instance
I20260806 06:33:53.396018 7341 external_mini_cluster.cc:1428] Setting key f25ab516fe1f19a32328ff3be396591a
I20260806 06:33:53.396418 30060 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8e48a8a8158c4faeaf4beb84cddceaa7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e48a8a8158c4faeaf4beb84cddceaa7" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 33109 } } }
I20260806 06:33:53.396610 30060 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7 [sys.catalog]: This master's current role is: LEADER
I20260806 06:33:53.397737 30062 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e48a8a8158c4faeaf4beb84cddceaa7. Latest consensus state: current_term: 1 leader_uuid: "8e48a8a8158c4faeaf4beb84cddceaa7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e48a8a8158c4faeaf4beb84cddceaa7" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 33109 } } }
I20260806 06:33:53.397929 30062 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7 [sys.catalog]: This master's current role is: LEADER
W20260806 06:33:53.398377 30087 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:33:53.398473 30087 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:33:53.398591 30088 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:33:53.398810 30088 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:33:53.400085 30088 catalog_manager.cc:1347] Generated new cluster ID: d3b25010ada7481c9edd95e8af43ff62
I20260806 06:33:53.400130 30088 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:33:53 dist-test-slave-6zbq krb5kdc[29932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998033, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:33:53.406472 30088 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:33:53.406929 30088 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:33:53.408490 30073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:53.396409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56314 (local address 127.7.43.126:33109)
0806 06:33:53.396582 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:53.396591 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:53.396850 (+ 259us) server_negotiation.cc:407] Connection header received
0806 06:33:53.397055 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:53.397063 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:53.397121 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:53.397242 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:33:53.398142 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:53.399769 (+ 1627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:53.400839 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:53.401462 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:53.401562 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:53.404321 (+ 2759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:33:53.404326 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:33:53.404326 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:33:53.404337 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:33:53.406586 (+ 2249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:53.406994 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:53.406997 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:53.406997 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:53.407098 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:53.407299 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:53.407300 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:53.407300 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:53.407573 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:33:53.407711 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:53.408153 (+ 442us) server_negotiation.cc:299] Negotiation successful
0806 06:33:53.408294 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":83,"threads_started":1}
I20260806 06:33:53.410658 30088 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7: Generated new TSK 0
I20260806 06:33:53.410818 30088 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.43.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab.
Entry for principal kudu/127.7.43.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab.
I20260806 06:33:53.437193 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.65:0
--local_ip_for_outbound_sockets=127.7.43.65
--webserver_interface=127.7.43.65
--webserver_port=0
--tserver_master_addrs=127.7.43.126:33109
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab
--principal=kudu/127.7.43.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:40943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:33:53.516563 30098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:33:53.516749 30098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:33:53.516772 30098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:33:53.517858 30098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:33:53.517896 30098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:33:53.517920 30098 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.65
I20260806 06:33:53.519330 30098 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:40943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab
--principal=kudu/127.7.43.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.43.65
--webserver_port=0
--tserver_master_addrs=127.7.43.126:33109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30098
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.65
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:33:53.519534 30098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:33:53.519742 30098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:33:53.524688 30111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:33:53.524749 30098 server_base.cc:1034] running on GCE node
W20260806 06:33:53.524667 30108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:33:53.524688 30109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:33:53.525055 30098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:33:53.525254 30098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:33:53.526382 30098 hybrid_clock.cc:648] HybridClock initialized: now 1785998033526372 us; error 25 us; skew 500 ppm
Aug 06 06:33:53 dist-test-slave-6zbq krb5kdc[29932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998033, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:33:53.528672 30098 init.cc:377] Logged in from keytab as kudu/127.7.43.65@KRBTEST.COM (short username kudu)
I20260806 06:33:53.529021 30098 webserver.cc:460] Webserver started at http://127.7.43.65:44281/ using document root <none> and password file <none>
I20260806 06:33:53.529201 30098 fs_manager.cc:359] Metadata directory not provided
I20260806 06:33:53.529237 30098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:33:53.529321 30098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:33:53.530202 30098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-0/data/instance:
uuid: "ac53c77c4c2e4eae82362a041454a59f"
format_stamp: "Formatted at 2026-08-06 06:33:53 on dist-test-slave-6zbq"
server_key: "d0002476731721625c938e16e5d01a0c"
server_key_iv: "71847b749b16484bbcda29cb524aaf35"
server_key_version: "encryptionkey@0"
I20260806 06:33:53.530508 30098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-0/wal/instance:
uuid: "ac53c77c4c2e4eae82362a041454a59f"
format_stamp: "Formatted at 2026-08-06 06:33:53 on dist-test-slave-6zbq"
server_key: "d0002476731721625c938e16e5d01a0c"
server_key_iv: "71847b749b16484bbcda29cb524aaf35"
server_key_version: "encryptionkey@0"
I20260806 06:33:53.531670 30098 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:33:53.532248 30120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:53.532388 30098 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:33:53.532466 30098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-0/wal
uuid: "ac53c77c4c2e4eae82362a041454a59f"
format_stamp: "Formatted at 2026-08-06 06:33:53 on dist-test-slave-6zbq"
server_key: "d0002476731721625c938e16e5d01a0c"
server_key_iv: "71847b749b16484bbcda29cb524aaf35"
server_key_version: "encryptionkey@0"
I20260806 06:33:53.532518 30098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:33:53.547958 30098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:33:53.549513 30098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:33:53.549611 30098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:33:53.549770 30098 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:33:53.549990 30098 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:33:53.550019 30098 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:53.550036 30098 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:33:53.550052 30098 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:53.555475 30098 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.65:32807
I20260806 06:33:53.555533 30297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.65:32807 every 8 connection(s)
I20260806 06:33:53.555785 30098 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
Aug 06 06:33:53 dist-test-slave-6zbq krb5kdc[29932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998033, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.65@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:33:53.561359 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 30098
I20260806 06:33:53.561468 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-0/wal/instance
I20260806 06:33:53.561685 7341 external_mini_cluster.cc:1428] Setting key fa2a0e5c593d0b4876b9a43ccffa3026
I20260806 06:33:53.564203 30073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:53.556396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:58521 (local address 127.7.43.126:33109)
0806 06:33:53.556441 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:53.556443 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:53.556623 (+ 180us) server_negotiation.cc:407] Connection header received
0806 06:33:53.557029 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:53.557030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:53.557043 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:53.557077 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:33:53.557876 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:53.558406 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:53.559316 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:53.559605 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:53.559653 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:53.561598 (+ 1945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:33:53.561600 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:33:53.561601 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:33:53.561608 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:33:53.562846 (+ 1238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:53.563276 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:53.563277 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:53.563277 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:53.563323 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:53.563553 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:53.563553 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:53.563553 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:53.563649 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:33:53.563692 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:53.564033 (+ 341us) server_negotiation.cc:299] Negotiation successful
0806 06:33:53.564125 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 06:33:53.564595 30306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:53.556377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.65:58521)
0806 06:33:53.556522 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:53.556552 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 06:33:53.556948 (+ 396us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:33:53.557131 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:33:53.557133 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:33:53.557647 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:33:53.557813 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:53.557818 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:53.558474 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:53.558475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:53.559251 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:53.559254 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:53.559664 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:53.559665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:53.559770 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:53.560193 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:33:53.560201 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:33:53.561513 (+ 1312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:33:53.562925 (+ 1412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:33:53.562933 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:33:53.562935 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:33:53.563213 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:33:53.563378 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:33:53.563379 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:33:53.563379 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:33:53.563498 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:33:53.563697 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:33:53.563702 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:33:53.563841 (+ 139us) client_negotiation.cc:769] Sending connection context
0806 06:33:53.564285 (+ 444us) client_negotiation.cc:240] Negotiation successful
0806 06:33:53.564445 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":87,"threads_started":1}
I20260806 06:33:53.565001 30299 heartbeater.cc:347] Connected to a master server at 127.7.43.126:33109
I20260806 06:33:53.565085 30299 heartbeater.cc:464] Registering TS with master...
I20260806 06:33:53.565241 30299 heartbeater.cc:511] Master 127.7.43.126:33109 requested a full tablet report, sending...
I20260806 06:33:53.565764 29988 ts_manager.cc:194] Registered new tserver with Master: ac53c77c4c2e4eae82362a041454a59f (127.7.43.65:32807)
I20260806 06:33:53.566574 29988 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.65@KRBTEST.COM'} at 127.7.43.65:58521
WARNING: no policy specified for kudu/127.7.43.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab.
Entry for principal kudu/127.7.43.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab.
I20260806 06:33:53.584856 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.66:0
--local_ip_for_outbound_sockets=127.7.43.66
--webserver_interface=127.7.43.66
--webserver_port=0
--tserver_master_addrs=127.7.43.126:33109
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab
--principal=kudu/127.7.43.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:40943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:33:53.662397 30313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:33:53.662571 30313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:33:53.662601 30313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:33:53.663666 30313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:33:53.663709 30313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:33:53.663744 30313 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.66
I20260806 06:33:53.665102 30313 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:40943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab
--principal=kudu/127.7.43.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.43.66
--webserver_port=0
--tserver_master_addrs=127.7.43.126:33109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30313
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.66
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:33:53.665302 30313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:33:53.665493 30313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:33:53.670027 30326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:33:53.670034 30322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:33:53.670080 30323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:33:53.670140 30313 server_base.cc:1034] running on GCE node
I20260806 06:33:53.670437 30313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:33:53.670619 30313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:33:53.671761 30313 hybrid_clock.cc:648] HybridClock initialized: now 1785998033671758 us; error 32 us; skew 500 ppm
Aug 06 06:33:53 dist-test-slave-6zbq krb5kdc[29932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998033, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:33:53.674014 30313 init.cc:377] Logged in from keytab as kudu/127.7.43.66@KRBTEST.COM (short username kudu)
I20260806 06:33:53.674379 30313 webserver.cc:460] Webserver started at http://127.7.43.66:36801/ using document root <none> and password file <none>
I20260806 06:33:53.674548 30313 fs_manager.cc:359] Metadata directory not provided
I20260806 06:33:53.674587 30313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:33:53.674670 30313 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:33:53.675536 30313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-1/data/instance:
uuid: "f687bca681214cc79a90338958e7e828"
format_stamp: "Formatted at 2026-08-06 06:33:53 on dist-test-slave-6zbq"
server_key: "272b00e53b95fcb22e21f39eae054230"
server_key_iv: "2f8559e478d2a401bd2c11b9c9cbf464"
server_key_version: "encryptionkey@0"
I20260806 06:33:53.675810 30313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-1/wal/instance:
uuid: "f687bca681214cc79a90338958e7e828"
format_stamp: "Formatted at 2026-08-06 06:33:53 on dist-test-slave-6zbq"
server_key: "272b00e53b95fcb22e21f39eae054230"
server_key_iv: "2f8559e478d2a401bd2c11b9c9cbf464"
server_key_version: "encryptionkey@0"
I20260806 06:33:53.676895 30313 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:33:53.677501 30334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:53.677644 30313 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:33:53.677721 30313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-1/wal
uuid: "f687bca681214cc79a90338958e7e828"
format_stamp: "Formatted at 2026-08-06 06:33:53 on dist-test-slave-6zbq"
server_key: "272b00e53b95fcb22e21f39eae054230"
server_key_iv: "2f8559e478d2a401bd2c11b9c9cbf464"
server_key_version: "encryptionkey@0"
I20260806 06:33:53.677771 30313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:33:53.687434 30313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:33:53.689332 30313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:33:53.689431 30313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:33:53.689587 30313 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:33:53.689805 30313 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:33:53.689833 30313 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:53.689851 30313 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:33:53.689867 30313 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:53.695221 30313 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.66:39115
I20260806 06:33:53.695283 30497 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.66:39115 every 8 connection(s)
I20260806 06:33:53.695533 30313 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
I20260806 06:33:53.698961 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 30313
I20260806 06:33:53.699060 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-1/wal/instance
I20260806 06:33:53.699246 7341 external_mini_cluster.cc:1428] Setting key 0d012acf11bfd698040bd9b4842f681a
Aug 06 06:33:53 dist-test-slave-6zbq krb5kdc[29932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998033, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.66@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:33:53.704341 30073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:53.696125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:51929 (local address 127.7.43.126:33109)
0806 06:33:53.696176 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:53.696179 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:53.696377 (+ 198us) server_negotiation.cc:407] Connection header received
0806 06:33:53.696867 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:53.696869 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:53.696884 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:53.696926 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:33:53.697799 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:53.698291 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:53.699484 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:53.699681 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:53.699735 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:53.701855 (+ 2120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:33:53.701856 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:33:53.701857 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:33:53.701863 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:33:53.703139 (+ 1276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:53.703489 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:53.703491 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:53.703492 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:53.703557 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:53.703752 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:53.703753 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:53.703753 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:53.703881 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:33:53.703945 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:53.704158 (+ 213us) server_negotiation.cc:299] Negotiation successful
0806 06:33:53.704241 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 06:33:53.704391 30501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:53.696100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.66:51929)
0806 06:33:53.696254 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:53.696291 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 06:33:53.696750 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:33:53.696999 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:33:53.697003 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:33:53.697541 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:33:53.697710 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:53.697714 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:53.698397 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:53.698399 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:53.699402 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:53.699406 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:53.699748 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:53.699749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:53.699872 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:53.700285 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:33:53.700294 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:33:53.701775 (+ 1481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:33:53.703217 (+ 1442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:33:53.703223 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:33:53.703225 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:33:53.703420 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:33:53.703615 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:33:53.703615 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:33:53.703616 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:33:53.703700 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:33:53.703936 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:33:53.703939 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:33:53.704037 (+ 98us) client_negotiation.cc:769] Sending connection context
0806 06:33:53.704108 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 06:33:53.704227 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":77,"threads_started":1}
I20260806 06:33:53.704795 30499 heartbeater.cc:347] Connected to a master server at 127.7.43.126:33109
I20260806 06:33:53.704895 30499 heartbeater.cc:464] Registering TS with master...
I20260806 06:33:53.705070 30499 heartbeater.cc:511] Master 127.7.43.126:33109 requested a full tablet report, sending...
I20260806 06:33:53.705416 29988 ts_manager.cc:194] Registered new tserver with Master: f687bca681214cc79a90338958e7e828 (127.7.43.66:39115)
I20260806 06:33:53.705772 29988 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.66@KRBTEST.COM'} at 127.7.43.66:51929
WARNING: no policy specified for kudu/127.7.43.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab.
Entry for principal kudu/127.7.43.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab.
I20260806 06:33:53.722759 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.67:0
--local_ip_for_outbound_sockets=127.7.43.67
--webserver_interface=127.7.43.67
--webserver_port=0
--tserver_master_addrs=127.7.43.126:33109
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab
--principal=kudu/127.7.43.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:40943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:33:53.800544 30507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:33:53.800717 30507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:33:53.800737 30507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:33:53.801851 30507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:33:53.801889 30507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:33:53.801916 30507 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.67
I20260806 06:33:53.803328 30507 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:40943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab
--principal=kudu/127.7.43.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.43.67
--webserver_port=0
--tserver_master_addrs=127.7.43.126:33109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30507
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.67
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:33:53.803534 30507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:33:53.803719 30507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:33:53.808161 30516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:33:53.808164 30514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:33:53.808313 30507 server_base.cc:1034] running on GCE node
W20260806 06:33:53.808168 30518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:33:53.808589 30507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:33:53.808816 30507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:33:53.809955 30507 hybrid_clock.cc:648] HybridClock initialized: now 1785998033809942 us; error 27 us; skew 500 ppm
Aug 06 06:33:53 dist-test-slave-6zbq krb5kdc[29932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998033, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:33:53.812140 30507 init.cc:377] Logged in from keytab as kudu/127.7.43.67@KRBTEST.COM (short username kudu)
I20260806 06:33:53.812546 30507 webserver.cc:460] Webserver started at http://127.7.43.67:46467/ using document root <none> and password file <none>
I20260806 06:33:53.812742 30507 fs_manager.cc:359] Metadata directory not provided
I20260806 06:33:53.812788 30507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:33:53.812907 30507 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:33:53.813829 30507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-2/data/instance:
uuid: "fa4620d690aa48fd98f0d015919af71c"
format_stamp: "Formatted at 2026-08-06 06:33:53 on dist-test-slave-6zbq"
server_key: "2d4fb9cda3831e3ce0b052e141b01db3"
server_key_iv: "7ccd7702495d8a94d4b0ed26e1291135"
server_key_version: "encryptionkey@0"
I20260806 06:33:53.814141 30507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-2/wal/instance:
uuid: "fa4620d690aa48fd98f0d015919af71c"
format_stamp: "Formatted at 2026-08-06 06:33:53 on dist-test-slave-6zbq"
server_key: "2d4fb9cda3831e3ce0b052e141b01db3"
server_key_iv: "7ccd7702495d8a94d4b0ed26e1291135"
server_key_version: "encryptionkey@0"
I20260806 06:33:53.815362 30507 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:33:53.816000 30529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:53.816164 30507 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:33:53.816243 30507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-2/wal
uuid: "fa4620d690aa48fd98f0d015919af71c"
format_stamp: "Formatted at 2026-08-06 06:33:53 on dist-test-slave-6zbq"
server_key: "2d4fb9cda3831e3ce0b052e141b01db3"
server_key_iv: "7ccd7702495d8a94d4b0ed26e1291135"
server_key_version: "encryptionkey@0"
I20260806 06:33:53.816313 30507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:33:53.831574 30507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:33:53.833053 30507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:33:53.833174 30507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:33:53.833370 30507 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:33:53.833630 30507 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:33:53.833662 30507 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:53.833688 30507 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:33:53.833706 30507 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:33:53.839426 30507 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.67:46465
I20260806 06:33:53.839483 30706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.67:46465 every 8 connection(s)
I20260806 06:33:53.839735 30507 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
Aug 06 06:33:53 dist-test-slave-6zbq krb5kdc[29932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998033, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.67@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:33:53.847002 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 30507
I20260806 06:33:53.847110 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-2/wal/instance
I20260806 06:33:53.847285 30073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:53.840324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:41657 (local address 127.7.43.126:33109)
0806 06:33:53.840376 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:53.840378 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:53.840544 (+ 166us) server_negotiation.cc:407] Connection header received
0806 06:33:53.840955 (+ 411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:53.840957 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:53.840966 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:53.841011 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:33:53.841839 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:53.842320 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:53.843196 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:53.843370 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:53.843411 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:53.845125 (+ 1714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:33:53.845127 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:33:53.845127 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:33:53.845134 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:33:53.846158 (+ 1024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:53.846531 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:53.846531 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:53.846532 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:53.846575 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:53.846728 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:53.846729 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:53.846729 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:53.846838 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:33:53.846901 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:53.847107 (+ 206us) server_negotiation.cc:299] Negotiation successful
0806 06:33:53.847198 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:33:53.847293 7341 external_mini_cluster.cc:1428] Setting key 076593e789a93416ca9a78cb6b9a3799
I20260806 06:33:53.847393 30711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:53.840300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.67:41657)
0806 06:33:53.840438 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:53.840469 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 06:33:53.840883 (+ 414us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:33:53.841061 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:33:53.841064 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:33:53.841599 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:33:53.841773 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:53.841778 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:53.842397 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:53.842398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:53.843133 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:53.843136 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:53.843421 (+ 285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:53.843421 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:53.843508 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:53.843857 (+ 349us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:33:53.843863 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:33:53.845044 (+ 1181us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:33:53.846231 (+ 1187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:33:53.846237 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:33:53.846238 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:33:53.846478 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:33:53.846611 (+ 133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:33:53.846611 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:33:53.846612 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:33:53.846695 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:33:53.846891 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:33:53.846893 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:33:53.846991 (+ 98us) client_negotiation.cc:769] Sending connection context
0806 06:33:53.847071 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 06:33:53.847218 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":80,"threads_started":1}
I20260806 06:33:53.847895 30707 heartbeater.cc:347] Connected to a master server at 127.7.43.126:33109
I20260806 06:33:53.847976 30707 heartbeater.cc:464] Registering TS with master...
I20260806 06:33:53.848145 30707 heartbeater.cc:511] Master 127.7.43.126:33109 requested a full tablet report, sending...
I20260806 06:33:53.848474 29988 ts_manager.cc:194] Registered new tserver with Master: fa4620d690aa48fd98f0d015919af71c (127.7.43.67:46465)
I20260806 06:33:53.848816 29988 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.67@KRBTEST.COM'} at 127.7.43.67:41657
I20260806 06:33:53.858737 7341 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:33:53.865160 30073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:53.859941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56320 (local address 127.7.43.126:33109)
0806 06:33:53.859975 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:53.859977 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:53.860103 (+ 126us) server_negotiation.cc:407] Connection header received
0806 06:33:53.860163 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:53.860163 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:53.860186 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:53.860229 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:33:53.860754 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:53.861216 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:53.861914 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:53.862091 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:53.862142 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:53.863228 (+ 1086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:33:53.863229 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:33:53.863230 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:33:53.863235 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:33:53.864160 (+ 925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:53.864464 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:53.864465 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:53.864465 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:53.864514 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:33:53.864668 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:33:53.864669 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:33:53.864669 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:33:53.864801 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:33:53.864886 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:53.865008 (+ 122us) server_negotiation.cc:299] Negotiation successful
0806 06:33:53.865076 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260806 06:33:53.866447 29988 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56320:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:33:53.867918 29988 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:33:53.873677 30720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:53.869812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.67:46465 (local address 127.0.0.1:53792)
0806 06:33:53.869945 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:53.869987 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 06:33:53.870049 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:33:53.870651 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:33:53.870654 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:33:53.870656 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:33:53.870717 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:53.870719 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:53.871548 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:53.871549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:53.872608 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:53.872611 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:53.873399 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:53.873401 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:53.873478 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:53.873491 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 06:33:53.873556 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 06:33:53.873607 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":62,"threads_started":1}
I20260806 06:33:53.873689 30726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:53.870146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53792 (local address 127.7.43.67:46465)
0806 06:33:53.870334 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:53.870374 (+ 40us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:53.870384 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:33:53.870417 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:53.870418 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:53.870430 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:53.870501 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:33:53.870806 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:53.871454 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:53.872700 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:53.873329 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:53.873386 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:53.873442 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:53.873577 (+ 135us) server_negotiation.cc:299] Negotiation successful
0806 06:33:53.873615 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":105,"threads_started":1}
I20260806 06:33:53.873909 30725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:53.870010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.66:39115 (local address 127.0.0.1:44092)
0806 06:33:53.870128 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:53.870138 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:33:53.870185 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:33:53.870770 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:33:53.870771 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:33:53.870773 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:33:53.870800 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:53.870801 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:53.872055 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:53.872056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:53.872884 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:53.872886 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:53.873708 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:53.873710 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:53.873768 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:53.873773 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:33:53.873803 (+ 30us) client_negotiation.cc:240] Negotiation successful
0806 06:33:53.873839 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"mutex_wait_us":2,"thread_start_us":75,"threads_started":1}
I20260806 06:33:53.874179 30727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:53.870417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44092 (local address 127.7.43.66:39115)
0806 06:33:53.870608 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:53.870611 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:53.870619 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:33:53.870652 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:53.870654 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:53.870666 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:53.870867 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:33:53.870935 (+ 68us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:53.871942 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:53.872954 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:53.873598 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:53.873974 (+ 376us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:53.874041 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:53.874065 (+ 24us) server_negotiation.cc:299] Negotiation successful
0806 06:33:53.874105 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":89,"threads_started":1}
I20260806 06:33:53.874634 30604 tablet_service.cc:1467] Processing CreateTablet for tablet de299da8557248898e514004b7668050 (DEFAULT_TABLE table=test-table [id=fc5637c21d8e4528ad4718d9d531376d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:33:53.874773 30394 tablet_service.cc:1467] Processing CreateTablet for tablet de299da8557248898e514004b7668050 (DEFAULT_TABLE table=test-table [id=fc5637c21d8e4528ad4718d9d531376d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:33:53.874943 30604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de299da8557248898e514004b7668050. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:33:53.875104 30394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de299da8557248898e514004b7668050. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:33:53.877249 30731 tablet_bootstrap.cc:492] T de299da8557248898e514004b7668050 P fa4620d690aa48fd98f0d015919af71c: Bootstrap starting.
I20260806 06:33:53.877462 30721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:53.869814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:32807 (local address 127.0.0.1:53332)
0806 06:33:53.869966 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:53.869987 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 06:33:53.870049 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:33:53.870702 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:33:53.870703 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:33:53.870706 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:33:53.870751 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:53.870753 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:53.872018 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:53.872019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:53.873052 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:53.873055 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:53.873933 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:53.873934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:53.877297 (+ 3363us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:53.877302 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:33:53.877357 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 06:33:53.877396 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":82,"threads_started":1}
I20260806 06:33:53.877689 30722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:53.869938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53332 (local address 127.7.43.65:32807)
0806 06:33:53.870467 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:53.870472 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:53.870484 (+ 12us) server_negotiation.cc:407] Connection header received
0806 06:33:53.870525 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:53.870527 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:53.870541 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:53.870621 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:33:53.870826 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:53.871914 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:53.873127 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:53.873796 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:53.873850 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:53.873902 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:53.877564 (+ 3662us) server_negotiation.cc:299] Negotiation successful
0806 06:33:53.877609 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":492,"thread_start_us":74,"threads_started":1}
I20260806 06:33:53.877789 30732 tablet_bootstrap.cc:492] T de299da8557248898e514004b7668050 P f687bca681214cc79a90338958e7e828: Bootstrap starting.
I20260806 06:33:53.877988 30731 tablet_bootstrap.cc:654] T de299da8557248898e514004b7668050 P fa4620d690aa48fd98f0d015919af71c: Neither blocks nor log segments found. Creating new log.
I20260806 06:33:53.878489 30732 tablet_bootstrap.cc:654] T de299da8557248898e514004b7668050 P f687bca681214cc79a90338958e7e828: Neither blocks nor log segments found. Creating new log.
I20260806 06:33:53.878538 30731 log.cc:824] T de299da8557248898e514004b7668050 P fa4620d690aa48fd98f0d015919af71c: Log is configured to *not* fsync() on all Append() calls
I20260806 06:33:53.878466 30194 tablet_service.cc:1467] Processing CreateTablet for tablet de299da8557248898e514004b7668050 (DEFAULT_TABLE table=test-table [id=fc5637c21d8e4528ad4718d9d531376d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:33:53.878849 30194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de299da8557248898e514004b7668050. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:33:53.879038 30732 log.cc:824] T de299da8557248898e514004b7668050 P f687bca681214cc79a90338958e7e828: Log is configured to *not* fsync() on all Append() calls
I20260806 06:33:53.879231 30731 tablet_bootstrap.cc:492] T de299da8557248898e514004b7668050 P fa4620d690aa48fd98f0d015919af71c: No bootstrap required, opened a new log
I20260806 06:33:53.879374 30731 ts_tablet_manager.cc:1397] T de299da8557248898e514004b7668050 P fa4620d690aa48fd98f0d015919af71c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:33:53.879786 30732 tablet_bootstrap.cc:492] T de299da8557248898e514004b7668050 P f687bca681214cc79a90338958e7e828: No bootstrap required, opened a new log
I20260806 06:33:53.879858 30732 ts_tablet_manager.cc:1397] T de299da8557248898e514004b7668050 P f687bca681214cc79a90338958e7e828: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 06:33:53.880896 30731 raft_consensus.cc:348] T de299da8557248898e514004b7668050 P fa4620d690aa48fd98f0d015919af71c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4620d690aa48fd98f0d015919af71c" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 46465 } } peers { permanent_uuid: "ac53c77c4c2e4eae82362a041454a59f" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 32807 } } peers { permanent_uuid: "f687bca681214cc79a90338958e7e828" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 39115 } }
I20260806 06:33:53.881063 30731 raft_consensus.cc:374] T de299da8557248898e514004b7668050 P fa4620d690aa48fd98f0d015919af71c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:33:53.881147 30731 raft_consensus.cc:729] T de299da8557248898e514004b7668050 P fa4620d690aa48fd98f0d015919af71c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa4620d690aa48fd98f0d015919af71c, State: Initialized, Role: FOLLOWER
I20260806 06:33:53.881306 30731 consensus_queue.cc:260] T de299da8557248898e514004b7668050 P fa4620d690aa48fd98f0d015919af71c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4620d690aa48fd98f0d015919af71c" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 46465 } } peers { permanent_uuid: "ac53c77c4c2e4eae82362a041454a59f" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 32807 } } peers { permanent_uuid: "f687bca681214cc79a90338958e7e828" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 39115 } }
I20260806 06:33:53.881662 30731 ts_tablet_manager.cc:1428] T de299da8557248898e514004b7668050 P fa4620d690aa48fd98f0d015919af71c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:33:53.881685 30707 heartbeater.cc:503] Master 127.7.43.126:33109 was elected leader, sending a full tablet report...
I20260806 06:33:53.881806 30732 raft_consensus.cc:348] T de299da8557248898e514004b7668050 P f687bca681214cc79a90338958e7e828 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4620d690aa48fd98f0d015919af71c" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 46465 } } peers { permanent_uuid: "ac53c77c4c2e4eae82362a041454a59f" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 32807 } } peers { permanent_uuid: "f687bca681214cc79a90338958e7e828" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 39115 } }
I20260806 06:33:53.882037 30732 raft_consensus.cc:374] T de299da8557248898e514004b7668050 P f687bca681214cc79a90338958e7e828 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:33:53.882089 30736 tablet_bootstrap.cc:492] T de299da8557248898e514004b7668050 P ac53c77c4c2e4eae82362a041454a59f: Bootstrap starting.
I20260806 06:33:53.882129 30732 raft_consensus.cc:729] T de299da8557248898e514004b7668050 P f687bca681214cc79a90338958e7e828 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f687bca681214cc79a90338958e7e828, State: Initialized, Role: FOLLOWER
I20260806 06:33:53.882246 30732 consensus_queue.cc:260] T de299da8557248898e514004b7668050 P f687bca681214cc79a90338958e7e828 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4620d690aa48fd98f0d015919af71c" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 46465 } } peers { permanent_uuid: "ac53c77c4c2e4eae82362a041454a59f" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 32807 } } peers { permanent_uuid: "f687bca681214cc79a90338958e7e828" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 39115 } }
I20260806 06:33:53.882552 30732 ts_tablet_manager.cc:1428] T de299da8557248898e514004b7668050 P f687bca681214cc79a90338958e7e828: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 06:33:53.882588 30499 heartbeater.cc:503] Master 127.7.43.126:33109 was elected leader, sending a full tablet report...
I20260806 06:33:53.882772 30736 tablet_bootstrap.cc:654] T de299da8557248898e514004b7668050 P ac53c77c4c2e4eae82362a041454a59f: Neither blocks nor log segments found. Creating new log.
I20260806 06:33:53.883273 30736 log.cc:824] T de299da8557248898e514004b7668050 P ac53c77c4c2e4eae82362a041454a59f: Log is configured to *not* fsync() on all Append() calls
I20260806 06:33:53.883862 30736 tablet_bootstrap.cc:492] T de299da8557248898e514004b7668050 P ac53c77c4c2e4eae82362a041454a59f: No bootstrap required, opened a new log
I20260806 06:33:53.883917 30736 ts_tablet_manager.cc:1397] T de299da8557248898e514004b7668050 P ac53c77c4c2e4eae82362a041454a59f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 06:33:53.885154 30736 raft_consensus.cc:348] T de299da8557248898e514004b7668050 P ac53c77c4c2e4eae82362a041454a59f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4620d690aa48fd98f0d015919af71c" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 46465 } } peers { permanent_uuid: "ac53c77c4c2e4eae82362a041454a59f" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 32807 } } peers { permanent_uuid: "f687bca681214cc79a90338958e7e828" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 39115 } }
I20260806 06:33:53.885296 30736 raft_consensus.cc:374] T de299da8557248898e514004b7668050 P ac53c77c4c2e4eae82362a041454a59f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:33:53.885329 30736 raft_consensus.cc:729] T de299da8557248898e514004b7668050 P ac53c77c4c2e4eae82362a041454a59f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac53c77c4c2e4eae82362a041454a59f, State: Initialized, Role: FOLLOWER
I20260806 06:33:53.885426 30736 consensus_queue.cc:260] T de299da8557248898e514004b7668050 P ac53c77c4c2e4eae82362a041454a59f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4620d690aa48fd98f0d015919af71c" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 46465 } } peers { permanent_uuid: "ac53c77c4c2e4eae82362a041454a59f" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 32807 } } peers { permanent_uuid: "f687bca681214cc79a90338958e7e828" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 39115 } }
I20260806 06:33:53.885686 30736 ts_tablet_manager.cc:1428] T de299da8557248898e514004b7668050 P ac53c77c4c2e4eae82362a041454a59f: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 06:33:53.885799 30299 heartbeater.cc:503] Master 127.7.43.126:33109 was elected leader, sending a full tablet report...
I20260806 06:33:53.920491 30738 raft_consensus.cc:482] T de299da8557248898e514004b7668050 P f687bca681214cc79a90338958e7e828 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:33:53.920581 30738 raft_consensus.cc:504] T de299da8557248898e514004b7668050 P f687bca681214cc79a90338958e7e828 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4620d690aa48fd98f0d015919af71c" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 46465 } } peers { permanent_uuid: "ac53c77c4c2e4eae82362a041454a59f" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 32807 } } peers { permanent_uuid: "f687bca681214cc79a90338958e7e828" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 39115 } }
I20260806 06:33:53.920902 30738 leader_election.cc:288] T de299da8557248898e514004b7668050 P f687bca681214cc79a90338958e7e828 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa4620d690aa48fd98f0d015919af71c (127.7.43.67:46465), ac53c77c4c2e4eae82362a041454a59f (127.7.43.65:32807)
I20260806 06:33:53.923548 30722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:53.921173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:52839 (local address 127.7.43.65:32807)
0806 06:33:53.921223 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:53.921225 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:53.921255 (+ 30us) server_negotiation.cc:407] Connection header received
0806 06:33:53.921317 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:53.921318 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:53.921328 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:53.921377 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:33:53.921517 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:53.921967 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:53.922895 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:53.923257 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:53.923316 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:53.923326 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:53.923454 (+ 128us) server_negotiation.cc:299] Negotiation successful
0806 06:33:53.923486 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 06:33:53.923552 30726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:53.921136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:48607 (local address 127.7.43.67:46465)
0806 06:33:53.921237 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:53.921239 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:53.921246 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:33:53.921270 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:53.921270 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:53.921281 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:53.921324 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:33:53.921498 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:53.921955 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:53.922881 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:53.923246 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:53.923313 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:53.923323 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:53.923453 (+ 130us) server_negotiation.cc:299] Negotiation successful
0806 06:33:53.923488 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 06:33:53.923563 30501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:53.921031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.67:46465 (local address 127.7.43.66:48607)
0806 06:33:53.921086 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:53.921112 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 06:33:53.921155 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:33:53.921369 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:33:53.921370 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:33:53.921371 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:33:53.921434 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:53.921436 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:53.922031 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:53.922032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:53.922807 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:53.922809 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:53.923323 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:53.923324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:53.923393 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:53.923396 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:33:53.923440 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 06:33:53.923479 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 06:33:53.923574 30743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:53.921037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:32807 (local address 127.7.43.66:52839)
0806 06:33:53.921203 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:53.921214 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:33:53.921266 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:33:53.921387 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:33:53.921388 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:33:53.921390 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:33:53.921456 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:53.921458 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:53.922036 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:53.922037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:53.922821 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:33:53.922823 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:33:53.923328 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:33:53.923329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:33:53.923394 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:53.923397 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:33:53.923443 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 06:33:53.923483 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":8,"thread_start_us":90,"threads_started":1}
I20260806 06:33:53.923889 30642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de299da8557248898e514004b7668050" candidate_uuid: "f687bca681214cc79a90338958e7e828" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa4620d690aa48fd98f0d015919af71c" is_pre_election: true
I20260806 06:33:53.924027 30642 raft_consensus.cc:2454] T de299da8557248898e514004b7668050 P fa4620d690aa48fd98f0d015919af71c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f687bca681214cc79a90338958e7e828 in term 0.
I20260806 06:33:53.924146 30219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de299da8557248898e514004b7668050" candidate_uuid: "f687bca681214cc79a90338958e7e828" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac53c77c4c2e4eae82362a041454a59f" is_pre_election: true
I20260806 06:33:53.924220 30336 leader_election.cc:302] T de299da8557248898e514004b7668050 P f687bca681214cc79a90338958e7e828 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f687bca681214cc79a90338958e7e828, fa4620d690aa48fd98f0d015919af71c; no voters:
I20260806 06:33:53.924281 30219 raft_consensus.cc:2454] T de299da8557248898e514004b7668050 P ac53c77c4c2e4eae82362a041454a59f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f687bca681214cc79a90338958e7e828 in term 0.
I20260806 06:33:53.924348 30738 raft_consensus.cc:2781] T de299da8557248898e514004b7668050 P f687bca681214cc79a90338958e7e828 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:33:53.924412 30738 raft_consensus.cc:482] T de299da8557248898e514004b7668050 P f687bca681214cc79a90338958e7e828 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:33:53.924428 30738 raft_consensus.cc:3037] T de299da8557248898e514004b7668050 P f687bca681214cc79a90338958e7e828 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:33:53.925066 30738 raft_consensus.cc:504] T de299da8557248898e514004b7668050 P f687bca681214cc79a90338958e7e828 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4620d690aa48fd98f0d015919af71c" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 46465 } } peers { permanent_uuid: "ac53c77c4c2e4eae82362a041454a59f" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 32807 } } peers { permanent_uuid: "f687bca681214cc79a90338958e7e828" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 39115 } }
I20260806 06:33:53.925181 30738 leader_election.cc:288] T de299da8557248898e514004b7668050 P f687bca681214cc79a90338958e7e828 [CANDIDATE]: Term 1 election: Requested vote from peers fa4620d690aa48fd98f0d015919af71c (127.7.43.67:46465), ac53c77c4c2e4eae82362a041454a59f (127.7.43.65:32807)
I20260806 06:33:53.925313 30642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de299da8557248898e514004b7668050" candidate_uuid: "f687bca681214cc79a90338958e7e828" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa4620d690aa48fd98f0d015919af71c"
I20260806 06:33:53.925334 30219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de299da8557248898e514004b7668050" candidate_uuid: "f687bca681214cc79a90338958e7e828" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac53c77c4c2e4eae82362a041454a59f"
I20260806 06:33:53.925380 30642 raft_consensus.cc:3037] T de299da8557248898e514004b7668050 P fa4620d690aa48fd98f0d015919af71c [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:33:53.925388 30219 raft_consensus.cc:3037] T de299da8557248898e514004b7668050 P ac53c77c4c2e4eae82362a041454a59f [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:33:53.926004 30642 raft_consensus.cc:2454] T de299da8557248898e514004b7668050 P fa4620d690aa48fd98f0d015919af71c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f687bca681214cc79a90338958e7e828 in term 1.
I20260806 06:33:53.926024 30219 raft_consensus.cc:2454] T de299da8557248898e514004b7668050 P ac53c77c4c2e4eae82362a041454a59f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f687bca681214cc79a90338958e7e828 in term 1.
I20260806 06:33:53.926182 30336 leader_election.cc:302] T de299da8557248898e514004b7668050 P f687bca681214cc79a90338958e7e828 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f687bca681214cc79a90338958e7e828, fa4620d690aa48fd98f0d015919af71c; no voters:
I20260806 06:33:53.926306 30738 raft_consensus.cc:2781] T de299da8557248898e514004b7668050 P f687bca681214cc79a90338958e7e828 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:33:53.926461 30738 raft_consensus.cc:686] T de299da8557248898e514004b7668050 P f687bca681214cc79a90338958e7e828 [term 1 LEADER]: Becoming Leader. State: Replica: f687bca681214cc79a90338958e7e828, State: Running, Role: LEADER
I20260806 06:33:53.926582 30738 consensus_queue.cc:237] T de299da8557248898e514004b7668050 P f687bca681214cc79a90338958e7e828 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4620d690aa48fd98f0d015919af71c" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 46465 } } peers { permanent_uuid: "ac53c77c4c2e4eae82362a041454a59f" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 32807 } } peers { permanent_uuid: "f687bca681214cc79a90338958e7e828" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 39115 } }
I20260806 06:33:53.927294 29988 catalog_manager.cc:5495] T de299da8557248898e514004b7668050 P f687bca681214cc79a90338958e7e828 reported cstate change: term changed from 0 to 1, leader changed from <none> to f687bca681214cc79a90338958e7e828 (127.7.43.66). New cstate: current_term: 1 leader_uuid: "f687bca681214cc79a90338958e7e828" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4620d690aa48fd98f0d015919af71c" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 46465 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac53c77c4c2e4eae82362a041454a59f" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 32807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f687bca681214cc79a90338958e7e828" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 39115 } health_report { overall_health: HEALTHY } } }
I20260806 06:33:53.944926 30727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:53.943082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44108 (local address 127.7.43.66:39115)
0806 06:33:53.943144 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 06:33:53.943145 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:33:53.943152 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:33:53.943177 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:33:53.943178 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:33:53.943193 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:33:53.943248 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:33:53.943349 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:53.943759 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:53.944393 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:33:53.944558 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:33:53.944597 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:33:53.944701 (+ 104us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:33:53.944786 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:33:53.944837 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 06:33:53.944866 (+ 29us) server_negotiation.cc:299] Negotiation successful
0806 06:33:53.944881 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260806 06:33:53.946081 30500 tablet.cc:2369] T de299da8557248898e514004b7668050 P f687bca681214cc79a90338958e7e828: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:33:53.946412 30219 raft_consensus.cc:1261] T de299da8557248898e514004b7668050 P ac53c77c4c2e4eae82362a041454a59f [term 1 FOLLOWER]: Refusing update from remote peer f687bca681214cc79a90338958e7e828: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:33:53.946429 30642 raft_consensus.cc:1261] T de299da8557248898e514004b7668050 P fa4620d690aa48fd98f0d015919af71c [term 1 FOLLOWER]: Refusing update from remote peer f687bca681214cc79a90338958e7e828: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:33:53.946708 30745 consensus_queue.cc:1035] T de299da8557248898e514004b7668050 P f687bca681214cc79a90338958e7e828 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa4620d690aa48fd98f0d015919af71c" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 46465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:33:53.946818 30738 consensus_queue.cc:1035] T de299da8557248898e514004b7668050 P f687bca681214cc79a90338958e7e828 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac53c77c4c2e4eae82362a041454a59f" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 32807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:33:53.948948 30750 mvcc.cc:205] Tried to move back new op lower bound from 7315447947042349056 to 7315447946963894272. Current Snapshot: MvccSnapshot[applied={T|T < 7315447947042349056}]
I20260806 06:33:53.949405 30752 mvcc.cc:205] Tried to move back new op lower bound from 7315447947042349056 to 7315447946963894272. Current Snapshot: MvccSnapshot[applied={T|T < 7315447947042349056}]
I20260806 06:33:53.949703 30749 mvcc.cc:205] Tried to move back new op lower bound from 7315447947042349056 to 7315447946963894272. Current Snapshot: MvccSnapshot[applied={T|T < 7315447947042349056 or (T in {7315447947042349056})}]
I20260806 06:33:53.951184 29988 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56320:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:33:53.951262 29988 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56320:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:33:53.951892 29988 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7: Sending DeleteTablet for 3 replicas of tablet de299da8557248898e514004b7668050
I20260806 06:33:53.952204 30604 tablet_service.cc:1514] Processing DeleteTablet for tablet de299da8557248898e514004b7668050 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:33:53 UTC) from {username='kudu', principal='kudu/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:53792
I20260806 06:33:53.952236 30394 tablet_service.cc:1514] Processing DeleteTablet for tablet de299da8557248898e514004b7668050 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:33:53 UTC) from {username='kudu', principal='kudu/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:44092
I20260806 06:33:53.952436 30763 tablet_replica.cc:330] T de299da8557248898e514004b7668050 P fa4620d690aa48fd98f0d015919af71c: stopping tablet replica
I20260806 06:33:53.952442 30194 tablet_service.cc:1514] Processing DeleteTablet for tablet de299da8557248898e514004b7668050 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:33:53 UTC) from {username='kudu', principal='kudu/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:53332
I20260806 06:33:53.952534 30763 raft_consensus.cc:2229] T de299da8557248898e514004b7668050 P fa4620d690aa48fd98f0d015919af71c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:33:53.952628 30763 raft_consensus.cc:2258] T de299da8557248898e514004b7668050 P fa4620d690aa48fd98f0d015919af71c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:33:53.952630 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 30098
I20260806 06:33:53.952654 30765 tablet_replica.cc:330] T de299da8557248898e514004b7668050 P ac53c77c4c2e4eae82362a041454a59f: stopping tablet replica
I20260806 06:33:53.952840 30765 raft_consensus.cc:2229] T de299da8557248898e514004b7668050 P ac53c77c4c2e4eae82362a041454a59f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:33:53.952993 30765 raft_consensus.cc:2258] T de299da8557248898e514004b7668050 P ac53c77c4c2e4eae82362a041454a59f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:33:53.953032 30763 ts_tablet_manager.cc:1905] T de299da8557248898e514004b7668050 P fa4620d690aa48fd98f0d015919af71c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:33:53.954789 30763 ts_tablet_manager.cc:1918] T de299da8557248898e514004b7668050 P fa4620d690aa48fd98f0d015919af71c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:33:53.954860 30763 log.cc:1196] T de299da8557248898e514004b7668050 P fa4620d690aa48fd98f0d015919af71c: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-2/wal/wals/de299da8557248898e514004b7668050
I20260806 06:33:53.955147 30763 ts_tablet_manager.cc:1939] T de299da8557248898e514004b7668050 P fa4620d690aa48fd98f0d015919af71c: Deleting consensus metadata
I20260806 06:33:53.955622 29969 catalog_manager.cc:4841] TS fa4620d690aa48fd98f0d015919af71c (127.7.43.67:46465): tablet de299da8557248898e514004b7668050 (table test-table [id=fc5637c21d8e4528ad4718d9d531376d]) successfully deleted
I20260806 06:33:53.955802 30762 tablet_replica.cc:330] T de299da8557248898e514004b7668050 P f687bca681214cc79a90338958e7e828: stopping tablet replica
I20260806 06:33:53.955899 30762 raft_consensus.cc:2229] T de299da8557248898e514004b7668050 P f687bca681214cc79a90338958e7e828 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:33:53.956032 30762 raft_consensus.cc:2258] T de299da8557248898e514004b7668050 P f687bca681214cc79a90338958e7e828 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:33:53.957846 30762 ts_tablet_manager.cc:1905] T de299da8557248898e514004b7668050 P f687bca681214cc79a90338958e7e828: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:33:53.959661 30762 ts_tablet_manager.cc:1918] T de299da8557248898e514004b7668050 P f687bca681214cc79a90338958e7e828: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:33:53.959722 30762 log.cc:1196] T de299da8557248898e514004b7668050 P f687bca681214cc79a90338958e7e828: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-1/wal/wals/de299da8557248898e514004b7668050
I20260806 06:33:53.959982 30762 ts_tablet_manager.cc:1939] T de299da8557248898e514004b7668050 P f687bca681214cc79a90338958e7e828: Deleting consensus metadata
I20260806 06:33:53.960316 29969 catalog_manager.cc:4841] TS f687bca681214cc79a90338958e7e828 (127.7.43.66:39115): tablet de299da8557248898e514004b7668050 (table test-table [id=fc5637c21d8e4528ad4718d9d531376d]) successfully deleted
W20260806 06:33:53.966472 29968 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.43.65:32807 (error 108)
I20260806 06:33:53.966872 30721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:33:53.966758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:32807 (local address 127.0.0.1:53340)
0806 06:33:53.966810 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 06:33:53.966843 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.65:32807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 06:33:53.966977 29968 catalog_manager.cc:4584] TS ac53c77c4c2e4eae82362a041454a59f (127.7.43.65:32807): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet de299da8557248898e514004b7668050: Network error: Client connection negotiation failed: client connection to 127.7.43.65:32807: connect: Connection refused (error 111)
I20260806 06:33:53.967103 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 30313
I20260806 06:33:53.974310 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 30507
I20260806 06:33:53.981305 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 29948
I20260806 06:34:13.989779 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:33109
--webserver_interface=127.7.43.126
--webserver_port=42165
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:40943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.43.126:33109
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:34:14.070556 30768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:34:14.070739 30768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:34:14.070760 30768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:34:14.071830 30768 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 06:34:14.071859 30768 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 06:34:14.071871 30768 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 06:34:14.071882 30768 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:34:14.071897 30768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:34:14.071909 30768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:34:14.071925 30768 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:34:14.071936 30768 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:34:14.073318 30768 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:40943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.43.126:33109
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:33109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.43.126
--webserver_port=42165
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30768
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:34:14.073488 30768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:34:14.073653 30768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:34:14.082055 30780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:34:14.082172 30778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:34:14.082132 30768 server_base.cc:1034] running on GCE node
W20260806 06:34:14.082055 30782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:34:14.082643 30768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:34:14.082931 30768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:34:14.084069 30768 hybrid_clock.cc:648] HybridClock initialized: now 1785998054084053 us; error 33 us; skew 500 ppm
Aug 06 06:34:14 dist-test-slave-6zbq krb5kdc[29932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998054, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:34:14.094767 30768 init.cc:377] Logged in from keytab as kudu/127.7.43.126@KRBTEST.COM (short username kudu)
I20260806 06:34:14.095240 30768 webserver.cc:460] Webserver started at http://127.7.43.126:42165/ using document root <none> and password file <none>
I20260806 06:34:14.095470 30768 fs_manager.cc:359] Metadata directory not provided
I20260806 06:34:14.095534 30768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:34:14.096796 30768 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:34:14.097384 30791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:14.097532 30768 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:14.097611 30768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "8e48a8a8158c4faeaf4beb84cddceaa7"
format_stamp: "Formatted at 2026-08-06 06:33:53 on dist-test-slave-6zbq"
server_key: "d8709f3cd43533890902d511c9bc7330"
server_key_iv: "57e178add804559622f849671220d387"
server_key_version: "encryptionkey@0"
I20260806 06:34:14.097983 30768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:34:14.118089 30768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:34:14.119777 30768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:34:14.119905 30768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:34:14.123688 30768 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.126:33109
I20260806 06:34:14.123724 30865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.126:33109 every 8 connection(s)
I20260806 06:34:14.124065 30768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:34:14.125547 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 30768
I20260806 06:34:14.125569 30868 sys_catalog.cc:263] Verifying existing consensus state
I20260806 06:34:14.125874 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.65:32807
--local_ip_for_outbound_sockets=127.7.43.65
--tserver_master_addrs=127.7.43.126:33109
--webserver_port=44281
--webserver_interface=127.7.43.65
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab
--principal=kudu/127.7.43.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:40943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:34:14.126447 30868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7: Bootstrap starting.
I20260806 06:34:14.134956 30868 log.cc:824] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7: Log is configured to *not* fsync() on all Append() calls
I20260806 06:34:14.136744 30868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:34:14.136996 30868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7: Bootstrap complete.
I20260806 06:34:14.139169 30868 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e48a8a8158c4faeaf4beb84cddceaa7" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 33109 } }
I20260806 06:34:14.139393 30868 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e48a8a8158c4faeaf4beb84cddceaa7, State: Initialized, Role: FOLLOWER
I20260806 06:34:14.139539 30868 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e48a8a8158c4faeaf4beb84cddceaa7" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 33109 } }
I20260806 06:34:14.139597 30868 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:34:14.139642 30868 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:34:14.139686 30868 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7 [term 1 FOLLOWER]: Advancing to term 2
I20260806 06:34:14.140666 30868 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e48a8a8158c4faeaf4beb84cddceaa7" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 33109 } }
I20260806 06:34:14.140753 30868 leader_election.cc:302] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e48a8a8158c4faeaf4beb84cddceaa7; no voters:
I20260806 06:34:14.140897 30868 leader_election.cc:288] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 06:34:14.140945 30875 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 06:34:14.141063 30875 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7 [term 2 LEADER]: Becoming Leader. State: Replica: 8e48a8a8158c4faeaf4beb84cddceaa7, State: Running, Role: LEADER
I20260806 06:34:14.141176 30868 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:34:14.141227 30875 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e48a8a8158c4faeaf4beb84cddceaa7" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 33109 } }
I20260806 06:34:14.141523 30875 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e48a8a8158c4faeaf4beb84cddceaa7. Latest consensus state: current_term: 2 leader_uuid: "8e48a8a8158c4faeaf4beb84cddceaa7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e48a8a8158c4faeaf4beb84cddceaa7" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 33109 } } }
I20260806 06:34:14.141583 30876 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "8e48a8a8158c4faeaf4beb84cddceaa7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e48a8a8158c4faeaf4beb84cddceaa7" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 33109 } } }
I20260806 06:34:14.141707 30876 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7 [sys.catalog]: This master's current role is: LEADER
I20260806 06:34:14.142035 30875 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7 [sys.catalog]: This master's current role is: LEADER
I20260806 06:34:14.142293 30887 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:34:14.143064 30887 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:34:14.143173 30887 catalog_manager.cc:1259] Loaded cluster ID: d3b25010ada7481c9edd95e8af43ff62
I20260806 06:34:14.143208 30887 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:34:14.144343 30887 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:34:14.144526 30887 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7: Loaded TSK: 0
I20260806 06:34:14.159104 30887 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7: Generated new TSK 1
I20260806 06:34:14.159202 30887 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 06:34:14.206735 30870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:34:14.206918 30870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:34:14.206951 30870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:34:14.208027 30870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:34:14.208067 30870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:34:14.208101 30870 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.65
I20260806 06:34:14.209494 30870 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:40943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab
--principal=kudu/127.7.43.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.65:32807
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.43.65
--webserver_port=44281
--tserver_master_addrs=127.7.43.126:33109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30870
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.65
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:34:14.209702 30870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:34:14.209892 30870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:34:14.215517 30910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:34:14.215595 30912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:34:14.215512 30909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:34:14.215771 30870 server_base.cc:1034] running on GCE node
I20260806 06:34:14.215929 30870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:34:14.216128 30870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:34:14.217267 30870 hybrid_clock.cc:648] HybridClock initialized: now 1785998054217253 us; error 30 us; skew 500 ppm
Aug 06 06:34:14 dist-test-slave-6zbq krb5kdc[29932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998054, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:34:14.219844 30870 init.cc:377] Logged in from keytab as kudu/127.7.43.65@KRBTEST.COM (short username kudu)
I20260806 06:34:14.220371 30870 webserver.cc:460] Webserver started at http://127.7.43.65:44281/ using document root <none> and password file <none>
I20260806 06:34:14.220606 30870 fs_manager.cc:359] Metadata directory not provided
I20260806 06:34:14.220697 30870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:34:14.222357 30870 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:34:14.223088 30923 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:14.223249 30870 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:34:14.223346 30870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-0/wal
uuid: "ac53c77c4c2e4eae82362a041454a59f"
format_stamp: "Formatted at 2026-08-06 06:33:53 on dist-test-slave-6zbq"
server_key: "d0002476731721625c938e16e5d01a0c"
server_key_iv: "71847b749b16484bbcda29cb524aaf35"
server_key_version: "encryptionkey@0"
I20260806 06:34:14.223740 30870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:34:14.243722 30870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:34:14.245900 30870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:34:14.246047 30870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:34:14.246284 30870 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:34:14.246824 30935 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 06:34:14.248164 30870 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 06:34:14.248225 30870 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 06:34:14.248256 30870 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 06:34:14.249085 30870 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 06:34:14.249131 30870 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 06:34:14.249181 30935 tablet_bootstrap.cc:492] T de299da8557248898e514004b7668050 P ac53c77c4c2e4eae82362a041454a59f: Bootstrap starting.
I20260806 06:34:14.256230 30870 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.65:32807
I20260806 06:34:14.256374 31089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.65:32807 every 8 connection(s)
I20260806 06:34:14.256685 30870 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
I20260806 06:34:14.259735 30935 log.cc:824] T de299da8557248898e514004b7668050 P ac53c77c4c2e4eae82362a041454a59f: Log is configured to *not* fsync() on all Append() calls
I20260806 06:34:14.261344 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 30870
I20260806 06:34:14.261636 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.66:39115
--local_ip_for_outbound_sockets=127.7.43.66
--tserver_master_addrs=127.7.43.126:33109
--webserver_port=36801
--webserver_interface=127.7.43.66
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab
--principal=kudu/127.7.43.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:40943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:34:14.261942 30935 tablet_bootstrap.cc:492] T de299da8557248898e514004b7668050 P ac53c77c4c2e4eae82362a041454a59f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:34:14.262193 30935 tablet_bootstrap.cc:492] T de299da8557248898e514004b7668050 P ac53c77c4c2e4eae82362a041454a59f: Bootstrap complete.
I20260806 06:34:14.263273 30935 ts_tablet_manager.cc:1397] T de299da8557248898e514004b7668050 P ac53c77c4c2e4eae82362a041454a59f: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.010s
I20260806 06:34:14.264509 30935 raft_consensus.cc:348] T de299da8557248898e514004b7668050 P ac53c77c4c2e4eae82362a041454a59f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4620d690aa48fd98f0d015919af71c" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 46465 } } peers { permanent_uuid: "ac53c77c4c2e4eae82362a041454a59f" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 32807 } } peers { permanent_uuid: "f687bca681214cc79a90338958e7e828" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 39115 } }
I20260806 06:34:14.264909 30935 raft_consensus.cc:729] T de299da8557248898e514004b7668050 P ac53c77c4c2e4eae82362a041454a59f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac53c77c4c2e4eae82362a041454a59f, State: Initialized, Role: FOLLOWER
I20260806 06:34:14.265105 30935 consensus_queue.cc:260] T de299da8557248898e514004b7668050 P ac53c77c4c2e4eae82362a041454a59f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4620d690aa48fd98f0d015919af71c" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 46465 } } peers { permanent_uuid: "ac53c77c4c2e4eae82362a041454a59f" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 32807 } } peers { permanent_uuid: "f687bca681214cc79a90338958e7e828" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 39115 } }
I20260806 06:34:14.265413 30935 ts_tablet_manager.cc:1428] T de299da8557248898e514004b7668050 P ac53c77c4c2e4eae82362a041454a59f: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
Aug 06 06:34:14 dist-test-slave-6zbq krb5kdc[29932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998054, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.65@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:34:14.269956 31093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:14.257473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.65:54533)
0806 06:34:14.257868 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:14.257926 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:14.258586 (+ 660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:14.259079 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:14.259084 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:14.259486 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:34:14.259742 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:14.259749 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:14.261658 (+ 1909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:14.261659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:14.262974 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:14.262978 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:14.263911 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:14.263913 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:14.264051 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:14.264607 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:34:14.264625 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:34:14.266255 (+ 1630us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:34:14.268324 (+ 2069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:34:14.268331 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:34:14.268333 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:34:14.268594 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:34:14.268849 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:34:14.268850 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:34:14.268850 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:34:14.268962 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:34:14.269372 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:34:14.269378 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:34:14.269514 (+ 136us) client_negotiation.cc:769] Sending connection context
0806 06:34:14.269605 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 06:34:14.269767 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":101,"threads_started":1}
I20260806 06:34:14.270589 31095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:14.257575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:54533 (local address 127.7.43.126:33109)
0806 06:34:14.257760 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:14.257773 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:14.258059 (+ 286us) server_negotiation.cc:407] Connection header received
0806 06:34:14.258823 (+ 764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:14.258830 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:14.258887 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:14.259020 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:34:14.259938 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:14.261478 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:14.263089 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:14.263698 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:14.263800 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:14.266382 (+ 2582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:34:14.266386 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:34:14.266386 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:34:14.266397 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:34:14.268238 (+ 1841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:34:14.268686 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:34:14.268689 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:34:14.268690 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:34:14.268790 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:34:14.269023 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:34:14.269023 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:34:14.269024 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:34:14.269271 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:34:14.269344 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:14.270250 (+ 906us) server_negotiation.cc:299] Negotiation successful
0806 06:34:14.270399 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":75,"threads_started":1}
I20260806 06:34:14.271219 31090 heartbeater.cc:347] Connected to a master server at 127.7.43.126:33109
I20260806 06:34:14.271318 31090 heartbeater.cc:464] Registering TS with master...
I20260806 06:34:14.271492 31090 heartbeater.cc:511] Master 127.7.43.126:33109 requested a full tablet report, sending...
I20260806 06:34:14.272300 30816 ts_manager.cc:194] Registered new tserver with Master: ac53c77c4c2e4eae82362a041454a59f (127.7.43.65:32807)
I20260806 06:34:14.273409 30816 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.65@KRBTEST.COM'} at 127.7.43.65:54533
I20260806 06:34:14.274500 31090 heartbeater.cc:503] Master 127.7.43.126:33109 was elected leader, sending a full tablet report...
I20260806 06:34:14.278276 31107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:14.273670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54070 (local address 127.7.43.65:32807)
0806 06:34:14.273804 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:14.273808 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:14.274164 (+ 356us) server_negotiation.cc:407] Connection header received
0806 06:34:14.274196 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:14.274198 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:14.274327 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:14.274420 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:34:14.274714 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:14.275835 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:14.277247 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:14.277942 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:14.278011 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:14.278060 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:14.278177 (+ 117us) server_negotiation.cc:299] Negotiation successful
0806 06:34:14.278212 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":68,"threads_started":1}
I20260806 06:34:14.278278 31105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:14.272918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:32807 (local address 127.0.0.1:54070)
0806 06:34:14.274003 (+ 1085us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:14.274036 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:14.274096 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:14.274482 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:14.274485 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:14.274487 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:34:14.274542 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:14.274544 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:14.275924 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:14.275925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:14.277166 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:14.277170 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:14.278015 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:14.278016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:14.278087 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:14.278099 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 06:34:14.278160 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 06:34:14.278207 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1059,"thread_start_us":597,"threads_started":1}
I20260806 06:34:14.278971 30996 tablet_service.cc:1514] Processing DeleteTablet for tablet de299da8557248898e514004b7668050 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:33:53 UTC) from {username='kudu', principal='kudu/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:54070
I20260806 06:34:14.279496 31109 tablet_replica.cc:330] stopping tablet replica
I20260806 06:34:14.279598 31109 raft_consensus.cc:2229] T de299da8557248898e514004b7668050 P ac53c77c4c2e4eae82362a041454a59f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:34:14.279665 31109 raft_consensus.cc:2258] T de299da8557248898e514004b7668050 P ac53c77c4c2e4eae82362a041454a59f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:34:14.279949 31109 ts_tablet_manager.cc:1905] T de299da8557248898e514004b7668050 P ac53c77c4c2e4eae82362a041454a59f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:34:14.282104 31109 ts_tablet_manager.cc:1918] T de299da8557248898e514004b7668050 P ac53c77c4c2e4eae82362a041454a59f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:34:14.282179 31109 log.cc:1196] T de299da8557248898e514004b7668050 P ac53c77c4c2e4eae82362a041454a59f: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-0/wal/wals/de299da8557248898e514004b7668050
I20260806 06:34:14.282416 31109 ts_tablet_manager.cc:1939] T de299da8557248898e514004b7668050 P ac53c77c4c2e4eae82362a041454a59f: Deleting consensus metadata
I20260806 06:34:14.282858 30793 catalog_manager.cc:4841] TS ac53c77c4c2e4eae82362a041454a59f (127.7.43.65:32807): tablet de299da8557248898e514004b7668050 (table test-table [id=fc5637c21d8e4528ad4718d9d531376d]) successfully deleted
W20260806 06:34:14.346884 31100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:34:14.347057 31100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:34:14.347078 31100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:34:14.348122 31100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:34:14.348156 31100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:34:14.348181 31100 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.66
I20260806 06:34:14.349532 31100 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:40943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab
--principal=kudu/127.7.43.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.66:39115
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.43.66
--webserver_port=36801
--tserver_master_addrs=127.7.43.126:33109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31100
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.66
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:34:14.349702 31100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:34:14.349862 31100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:34:14.354306 31119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:34:14.354287 31118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:34:14.354387 31100 server_base.cc:1034] running on GCE node
W20260806 06:34:14.354501 31122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:34:14.354689 31100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:34:14.354884 31100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:34:14.356014 31100 hybrid_clock.cc:648] HybridClock initialized: now 1785998054356000 us; error 27 us; skew 500 ppm
Aug 06 06:34:14 dist-test-slave-6zbq krb5kdc[29932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998054, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:34:14.358161 31100 init.cc:377] Logged in from keytab as kudu/127.7.43.66@KRBTEST.COM (short username kudu)
I20260806 06:34:14.358577 31100 webserver.cc:460] Webserver started at http://127.7.43.66:36801/ using document root <none> and password file <none>
I20260806 06:34:14.358783 31100 fs_manager.cc:359] Metadata directory not provided
I20260806 06:34:14.358844 31100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:34:14.360077 31100 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:34:14.360574 31144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:14.360721 31100 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:14.360809 31100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-1/wal
uuid: "f687bca681214cc79a90338958e7e828"
format_stamp: "Formatted at 2026-08-06 06:33:53 on dist-test-slave-6zbq"
server_key: "272b00e53b95fcb22e21f39eae054230"
server_key_iv: "2f8559e478d2a401bd2c11b9c9cbf464"
server_key_version: "encryptionkey@0"
I20260806 06:34:14.361088 31100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:34:14.371548 31100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:34:14.373407 31100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:34:14.373529 31100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:34:14.373703 31100 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:34:14.373956 31100 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:34:14.373987 31100 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:14.374013 31100 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:34:14.374037 31100 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:14.379516 31100 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.66:39115
I20260806 06:34:14.379554 31318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.66:39115 every 8 connection(s)
I20260806 06:34:14.379844 31100 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
Aug 06 06:34:14 dist-test-slave-6zbq krb5kdc[29932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998054, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.66@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:34:14.387573 31095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:14.380433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:51111 (local address 127.7.43.126:33109)
0806 06:34:14.380507 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:14.380509 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:14.380640 (+ 131us) server_negotiation.cc:407] Connection header received
0806 06:34:14.381056 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:14.381057 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:14.381067 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:14.381121 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:34:14.381922 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:14.382454 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:14.383324 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:14.383510 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:14.383553 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:14.385493 (+ 1940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:34:14.385494 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:34:14.385495 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:34:14.385503 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:34:14.386601 (+ 1098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:34:14.386926 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:34:14.386927 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:34:14.386927 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:34:14.386970 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:34:14.387140 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:34:14.387140 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:34:14.387141 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:34:14.387226 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:34:14.387268 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:14.387452 (+ 184us) server_negotiation.cc:299] Negotiation successful
0806 06:34:14.387509 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 06:34:14.387699 31322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:14.380403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.66:51111)
0806 06:34:14.380534 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:14.380566 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:14.380986 (+ 420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:14.381176 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:14.381179 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:14.381676 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:34:14.381857 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:14.381862 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:14.382515 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:14.382516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:14.383266 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:14.383268 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:14.383561 (+ 293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:14.383562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:14.383654 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:14.384054 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:34:14.384062 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:34:14.385408 (+ 1346us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:34:14.386673 (+ 1265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:34:14.386679 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:34:14.386681 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:34:14.386875 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:34:14.387013 (+ 138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:34:14.387013 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:34:14.387014 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:34:14.387100 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:34:14.387268 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:34:14.387270 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:34:14.387367 (+ 97us) client_negotiation.cc:769] Sending connection context
0806 06:34:14.387436 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 06:34:14.387539 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":75,"threads_started":1}
I20260806 06:34:14.388139 31319 heartbeater.cc:347] Connected to a master server at 127.7.43.126:33109
I20260806 06:34:14.388157 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 31100
I20260806 06:34:14.388229 31319 heartbeater.cc:464] Registering TS with master...
I20260806 06:34:14.388388 31319 heartbeater.cc:511] Master 127.7.43.126:33109 requested a full tablet report, sending...
I20260806 06:34:14.388423 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.67:46465
--local_ip_for_outbound_sockets=127.7.43.67
--tserver_master_addrs=127.7.43.126:33109
--webserver_port=46467
--webserver_interface=127.7.43.67
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab
--principal=kudu/127.7.43.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:40943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:34:14.388768 30816 ts_manager.cc:194] Registered new tserver with Master: f687bca681214cc79a90338958e7e828 (127.7.43.66:39115)
I20260806 06:34:14.389096 30816 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.66@KRBTEST.COM'} at 127.7.43.66:51111
W20260806 06:34:14.466812 31326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:34:14.466976 31326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:34:14.466997 31326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:34:14.468083 31326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:34:14.468122 31326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:34:14.468149 31326 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.67
I20260806 06:34:14.469792 31326 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:40943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab
--principal=kudu/127.7.43.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.67:46465
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.43.67
--webserver_port=46467
--tserver_master_addrs=127.7.43.126:33109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.67
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:34:14.470008 31326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:34:14.470225 31326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:34:14.474810 31333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:34:14.474835 31332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:34:14.474880 31326 server_base.cc:1034] running on GCE node
W20260806 06:34:14.474828 31338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:34:14.475165 31326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:34:14.475350 31326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:34:14.476485 31326 hybrid_clock.cc:648] HybridClock initialized: now 1785998054476472 us; error 27 us; skew 500 ppm
Aug 06 06:34:14 dist-test-slave-6zbq krb5kdc[29932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998054, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:34:14.478749 31326 init.cc:377] Logged in from keytab as kudu/127.7.43.67@KRBTEST.COM (short username kudu)
I20260806 06:34:14.479125 31326 webserver.cc:460] Webserver started at http://127.7.43.67:46467/ using document root <none> and password file <none>
I20260806 06:34:14.479321 31326 fs_manager.cc:359] Metadata directory not provided
I20260806 06:34:14.479377 31326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:34:14.480540 31326 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:34:14.481014 31354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:14.481165 31326 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:14.481254 31326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-2/wal
uuid: "fa4620d690aa48fd98f0d015919af71c"
format_stamp: "Formatted at 2026-08-06 06:33:53 on dist-test-slave-6zbq"
server_key: "2d4fb9cda3831e3ce0b052e141b01db3"
server_key_iv: "7ccd7702495d8a94d4b0ed26e1291135"
server_key_version: "encryptionkey@0"
I20260806 06:34:14.481515 31326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:34:14.488490 31326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:34:14.489971 31326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:34:14.490068 31326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:34:14.490234 31326 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:34:14.490485 31326 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:34:14.490517 31326 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:14.490535 31326 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:34:14.490553 31326 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:14.496292 31326 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.67:46465
I20260806 06:34:14.496326 31515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.67:46465 every 8 connection(s)
I20260806 06:34:14.496604 31326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
Aug 06 06:34:14 dist-test-slave-6zbq krb5kdc[29932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998054, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.67@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:34:14.503248 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 31326
I20260806 06:34:14.504704 31095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:14.497219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:48247 (local address 127.7.43.126:33109)
0806 06:34:14.497279 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:14.497281 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:14.497460 (+ 179us) server_negotiation.cc:407] Connection header received
0806 06:34:14.497854 (+ 394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:14.497855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:14.497864 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:14.497912 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:34:14.498750 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:14.499217 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:14.500113 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:14.500299 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:14.500357 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:14.502203 (+ 1846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:34:14.502204 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:34:14.502204 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:34:14.502210 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:34:14.503432 (+ 1222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:34:14.503850 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:34:14.503852 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:34:14.503852 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:34:14.503906 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:34:14.504149 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:34:14.504150 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:34:14.504151 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:34:14.504290 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:34:14.504364 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:14.504564 (+ 200us) server_negotiation.cc:299] Negotiation successful
0806 06:34:14.504640 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 06:34:14.504861 31518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:14.497192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.67:48247)
0806 06:34:14.497362 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:14.497393 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:14.497784 (+ 391us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:14.497965 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:14.497969 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:14.498508 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:34:14.498676 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:14.498680 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:14.499278 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:14.499279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:14.500026 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:14.500028 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:14.500366 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:14.500367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:14.500475 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:14.500853 (+ 378us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:34:14.500861 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:34:14.502116 (+ 1255us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:34:14.503509 (+ 1393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:34:14.503516 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:34:14.503518 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:34:14.503787 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:34:14.503968 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:34:14.503968 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:34:14.503968 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:34:14.504088 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:34:14.504352 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:34:14.504355 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:34:14.504453 (+ 98us) client_negotiation.cc:769] Sending connection context
0806 06:34:14.504523 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 06:34:14.504662 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":102,"threads_started":1}
I20260806 06:34:14.505249 31516 heartbeater.cc:347] Connected to a master server at 127.7.43.126:33109
I20260806 06:34:14.505330 31516 heartbeater.cc:464] Registering TS with master...
I20260806 06:34:14.505491 31516 heartbeater.cc:511] Master 127.7.43.126:33109 requested a full tablet report, sending...
I20260806 06:34:14.505844 30816 ts_manager.cc:194] Registered new tserver with Master: fa4620d690aa48fd98f0d015919af71c (127.7.43.67:46465)
I20260806 06:34:14.506177 30816 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.67@KRBTEST.COM'} at 127.7.43.67:48247
I20260806 06:34:14.509806 31521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:14.503590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38364 (local address 127.7.43.126:33109)
0806 06:34:14.503892 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:14.503894 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:14.503899 (+ 5us) server_negotiation.cc:407] Connection header received
0806 06:34:14.503917 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:14.503918 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:14.503925 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:14.503981 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:34:14.504614 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:14.505336 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:14.506204 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:14.506505 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:14.506567 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:14.507607 (+ 1040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:34:14.507608 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:34:14.507609 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:34:14.507616 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:34:14.508832 (+ 1216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:34:14.509133 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:34:14.509134 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:34:14.509135 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:34:14.509190 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:34:14.509348 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:34:14.509349 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:34:14.509349 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:34:14.509447 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:34:14.509610 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:14.509646 (+ 36us) server_negotiation.cc:299] Negotiation successful
0806 06:34:14.509719 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":60,"threads_started":1}
I20260806 06:34:14.510473 7341 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:34:14.512702 31521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:14.510761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38380 (local address 127.7.43.126:33109)
0806 06:34:14.510791 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:14.510793 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:14.510883 (+ 90us) server_negotiation.cc:407] Connection header received
0806 06:34:14.510936 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:14.510937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:14.510960 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:14.511016 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:34:14.511109 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:14.511516 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:14.512148 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:14.512317 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:14.512362 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:14.512454 (+ 92us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:34:14.512487 (+ 33us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0806 06:34:14.512664 (+ 177us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38380: token expired
Metrics: {"server-negotiator.queue_time_us":15}
W20260806 06:34:14.512745 31521 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38380: token expired
W20260806 06:34:14.513441 30713 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.7.43.126:33109)
I20260806 06:34:14.518141 31521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:14.513654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38396 (local address 127.7.43.126:33109)
0806 06:34:14.513683 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:14.513684 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:14.513692 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:34:14.513729 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:14.513729 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:14.513737 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:14.513777 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:34:14.514142 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:14.514610 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:14.515206 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:14.515362 (+ 156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:14.515400 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:14.516269 (+ 869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:34:14.516270 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:34:14.516270 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:34:14.516276 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:34:14.517396 (+ 1120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:34:14.517636 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:34:14.517636 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:34:14.517637 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:34:14.517678 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:34:14.517792 (+ 114us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:34:14.517793 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:34:14.517793 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:34:14.517890 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:34:14.517951 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:14.518022 (+ 71us) server_negotiation.cc:299] Negotiation successful
0806 06:34:14.518074 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
I20260806 06:34:14.519176 30816 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38396:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:34:14.520653 30816 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:34:14.522646 30996 tablet_service.cc:1467] Processing CreateTablet for tablet 3e1e75cee5f14a59a4c4618b8b02164b (DEFAULT_TABLE table=test-table [id=7f2f55fc63624a16ad3c20158c06c3c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:34:14.522840 30996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e1e75cee5f14a59a4c4618b8b02164b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:34:14.524205 30935 tablet_bootstrap.cc:492] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f: Bootstrap starting.
I20260806 06:34:14.524531 30935 tablet_bootstrap.cc:654] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f: Neither blocks nor log segments found. Creating new log.
I20260806 06:34:14.525072 30935 tablet_bootstrap.cc:492] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f: No bootstrap required, opened a new log
I20260806 06:34:14.525123 30935 ts_tablet_manager.cc:1397] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260806 06:34:14.525226 30935 raft_consensus.cc:348] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4620d690aa48fd98f0d015919af71c" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 46465 } } peers { permanent_uuid: "f687bca681214cc79a90338958e7e828" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 39115 } } peers { permanent_uuid: "ac53c77c4c2e4eae82362a041454a59f" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 32807 } }
I20260806 06:34:14.525296 30935 raft_consensus.cc:374] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:34:14.525344 30935 raft_consensus.cc:729] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac53c77c4c2e4eae82362a041454a59f, State: Initialized, Role: FOLLOWER
I20260806 06:34:14.525384 30935 consensus_queue.cc:260] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4620d690aa48fd98f0d015919af71c" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 46465 } } peers { permanent_uuid: "f687bca681214cc79a90338958e7e828" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 39115 } } peers { permanent_uuid: "ac53c77c4c2e4eae82362a041454a59f" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 32807 } }
I20260806 06:34:14.525470 30935 ts_tablet_manager.cc:1428] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:14.525993 31105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:14.522159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.67:46465 (local address 127.0.0.1:33144)
0806 06:34:14.522205 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:14.522215 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:14.522249 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:14.522597 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:14.522598 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:14.522599 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:34:14.522634 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:14.522635 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:14.523867 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:14.523868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:14.524763 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:14.524767 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:14.525783 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:14.525784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:14.525840 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:14.525844 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:34:14.525894 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 06:34:14.525932 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29,"mutex_wait_us":13}
I20260806 06:34:14.526008 31525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:14.522240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33144 (local address 127.7.43.67:46465)
0806 06:34:14.522433 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:14.522437 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:14.522446 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:34:14.522476 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:14.522478 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:14.522492 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:14.522695 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:34:14.522760 (+ 65us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:14.523776 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:14.524860 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:14.525694 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:14.525768 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:14.525825 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:14.525904 (+ 79us) server_negotiation.cc:299] Negotiation successful
0806 06:34:14.525940 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":121,"threads_started":1}
I20260806 06:34:14.526160 31528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:14.522260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.66:39115 (local address 127.0.0.1:51162)
0806 06:34:14.522527 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:14.522536 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:14.522577 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:14.522837 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:14.522837 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:14.522839 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:34:14.522873 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:14.522874 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:14.524035 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:14.524036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:14.525003 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:14.525007 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:14.525956 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:14.525957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:14.526010 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:14.526013 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:34:14.526057 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 06:34:14.526088 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"mutex_wait_us":3,"thread_start_us":110,"threads_started":1}
I20260806 06:34:14.526401 31529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:14.522397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51162 (local address 127.7.43.66:39115)
0806 06:34:14.522582 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:14.522586 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:14.522609 (+ 23us) server_negotiation.cc:407] Connection header received
0806 06:34:14.522650 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:14.522652 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:14.522666 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:14.522762 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:34:14.522962 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:14.523968 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:14.525108 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:14.525806 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:14.525868 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:14.525924 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:14.526262 (+ 338us) server_negotiation.cc:299] Negotiation successful
0806 06:34:14.526305 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":79,"threads_started":1}
I20260806 06:34:14.526662 31421 tablet_service.cc:1467] Processing CreateTablet for tablet 3e1e75cee5f14a59a4c4618b8b02164b (DEFAULT_TABLE table=test-table [id=7f2f55fc63624a16ad3c20158c06c3c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:34:14.526892 31421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e1e75cee5f14a59a4c4618b8b02164b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:34:14.529367 31531 tablet_bootstrap.cc:492] T 3e1e75cee5f14a59a4c4618b8b02164b P fa4620d690aa48fd98f0d015919af71c: Bootstrap starting.
I20260806 06:34:14.529923 31531 tablet_bootstrap.cc:654] T 3e1e75cee5f14a59a4c4618b8b02164b P fa4620d690aa48fd98f0d015919af71c: Neither blocks nor log segments found. Creating new log.
I20260806 06:34:14.529856 31224 tablet_service.cc:1467] Processing CreateTablet for tablet 3e1e75cee5f14a59a4c4618b8b02164b (DEFAULT_TABLE table=test-table [id=7f2f55fc63624a16ad3c20158c06c3c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:34:14.530099 31224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e1e75cee5f14a59a4c4618b8b02164b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:34:14.530476 31531 log.cc:824] T 3e1e75cee5f14a59a4c4618b8b02164b P fa4620d690aa48fd98f0d015919af71c: Log is configured to *not* fsync() on all Append() calls
I20260806 06:34:14.531149 31531 tablet_bootstrap.cc:492] T 3e1e75cee5f14a59a4c4618b8b02164b P fa4620d690aa48fd98f0d015919af71c: No bootstrap required, opened a new log
I20260806 06:34:14.531208 31531 ts_tablet_manager.cc:1397] T 3e1e75cee5f14a59a4c4618b8b02164b P fa4620d690aa48fd98f0d015919af71c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:34:14.532447 31531 raft_consensus.cc:348] T 3e1e75cee5f14a59a4c4618b8b02164b P fa4620d690aa48fd98f0d015919af71c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4620d690aa48fd98f0d015919af71c" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 46465 } } peers { permanent_uuid: "f687bca681214cc79a90338958e7e828" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 39115 } } peers { permanent_uuid: "ac53c77c4c2e4eae82362a041454a59f" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 32807 } }
I20260806 06:34:14.532575 31531 raft_consensus.cc:374] T 3e1e75cee5f14a59a4c4618b8b02164b P fa4620d690aa48fd98f0d015919af71c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:34:14.532608 31531 raft_consensus.cc:729] T 3e1e75cee5f14a59a4c4618b8b02164b P fa4620d690aa48fd98f0d015919af71c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa4620d690aa48fd98f0d015919af71c, State: Initialized, Role: FOLLOWER
I20260806 06:34:14.532699 31533 tablet_bootstrap.cc:492] T 3e1e75cee5f14a59a4c4618b8b02164b P f687bca681214cc79a90338958e7e828: Bootstrap starting.
I20260806 06:34:14.532737 31531 consensus_queue.cc:260] T 3e1e75cee5f14a59a4c4618b8b02164b P fa4620d690aa48fd98f0d015919af71c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4620d690aa48fd98f0d015919af71c" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 46465 } } peers { permanent_uuid: "f687bca681214cc79a90338958e7e828" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 39115 } } peers { permanent_uuid: "ac53c77c4c2e4eae82362a041454a59f" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 32807 } }
I20260806 06:34:14.533016 31531 ts_tablet_manager.cc:1428] T 3e1e75cee5f14a59a4c4618b8b02164b P fa4620d690aa48fd98f0d015919af71c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:34:14.533038 31516 heartbeater.cc:503] Master 127.7.43.126:33109 was elected leader, sending a full tablet report...
I20260806 06:34:14.533318 31533 tablet_bootstrap.cc:654] T 3e1e75cee5f14a59a4c4618b8b02164b P f687bca681214cc79a90338958e7e828: Neither blocks nor log segments found. Creating new log.
I20260806 06:34:14.533844 31533 log.cc:824] T 3e1e75cee5f14a59a4c4618b8b02164b P f687bca681214cc79a90338958e7e828: Log is configured to *not* fsync() on all Append() calls
I20260806 06:34:14.534566 31533 tablet_bootstrap.cc:492] T 3e1e75cee5f14a59a4c4618b8b02164b P f687bca681214cc79a90338958e7e828: No bootstrap required, opened a new log
I20260806 06:34:14.534621 31533 ts_tablet_manager.cc:1397] T 3e1e75cee5f14a59a4c4618b8b02164b P f687bca681214cc79a90338958e7e828: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:34:14.535843 31533 raft_consensus.cc:348] T 3e1e75cee5f14a59a4c4618b8b02164b P f687bca681214cc79a90338958e7e828 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4620d690aa48fd98f0d015919af71c" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 46465 } } peers { permanent_uuid: "f687bca681214cc79a90338958e7e828" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 39115 } } peers { permanent_uuid: "ac53c77c4c2e4eae82362a041454a59f" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 32807 } }
I20260806 06:34:14.535967 31533 raft_consensus.cc:374] T 3e1e75cee5f14a59a4c4618b8b02164b P f687bca681214cc79a90338958e7e828 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:34:14.535993 31533 raft_consensus.cc:729] T 3e1e75cee5f14a59a4c4618b8b02164b P f687bca681214cc79a90338958e7e828 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f687bca681214cc79a90338958e7e828, State: Initialized, Role: FOLLOWER
I20260806 06:34:14.536073 31533 consensus_queue.cc:260] T 3e1e75cee5f14a59a4c4618b8b02164b P f687bca681214cc79a90338958e7e828 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4620d690aa48fd98f0d015919af71c" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 46465 } } peers { permanent_uuid: "f687bca681214cc79a90338958e7e828" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 39115 } } peers { permanent_uuid: "ac53c77c4c2e4eae82362a041454a59f" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 32807 } }
I20260806 06:34:14.536296 31533 ts_tablet_manager.cc:1428] T 3e1e75cee5f14a59a4c4618b8b02164b P f687bca681214cc79a90338958e7e828: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:34:14.536345 31319 heartbeater.cc:503] Master 127.7.43.126:33109 was elected leader, sending a full tablet report...
I20260806 06:34:14.597479 31538 raft_consensus.cc:482] T 3e1e75cee5f14a59a4c4618b8b02164b P f687bca681214cc79a90338958e7e828 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:34:14.597561 31538 raft_consensus.cc:504] T 3e1e75cee5f14a59a4c4618b8b02164b P f687bca681214cc79a90338958e7e828 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4620d690aa48fd98f0d015919af71c" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 46465 } } peers { permanent_uuid: "f687bca681214cc79a90338958e7e828" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 39115 } } peers { permanent_uuid: "ac53c77c4c2e4eae82362a041454a59f" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 32807 } }
I20260806 06:34:14.597852 31538 leader_election.cc:288] T 3e1e75cee5f14a59a4c4618b8b02164b P f687bca681214cc79a90338958e7e828 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa4620d690aa48fd98f0d015919af71c (127.7.43.67:46465), ac53c77c4c2e4eae82362a041454a59f (127.7.43.65:32807)
I20260806 06:34:14.600440 31525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:14.598041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:36975 (local address 127.7.43.67:46465)
0806 06:34:14.598074 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:14.598075 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:14.598083 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:34:14.598105 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:14.598105 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:14.598115 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:14.598160 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:34:14.598280 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:14.598822 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:14.599718 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:14.600132 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:14.600185 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:14.600195 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:14.600351 (+ 156us) server_negotiation.cc:299] Negotiation successful
0806 06:34:14.600384 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 06:34:14.600477 31107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:14.598064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:52221 (local address 127.7.43.65:32807)
0806 06:34:14.598103 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:14.598105 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:14.598190 (+ 85us) server_negotiation.cc:407] Connection header received
0806 06:34:14.598240 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:14.598241 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:14.598251 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:14.598294 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:34:14.598444 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:14.598897 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:14.599765 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:14.600178 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:14.600235 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:14.600244 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:14.600379 (+ 135us) server_negotiation.cc:299] Negotiation successful
0806 06:34:14.600417 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:34:14.600468 31322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:14.597968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.67:46465 (local address 127.7.43.66:36975)
0806 06:34:14.598016 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:14.598026 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:14.598063 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:14.598161 (+ 98us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:14.598162 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:14.598163 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:34:14.598227 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:14.598229 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:14.598891 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:14.598892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:14.599631 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:14.599634 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:14.600215 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:14.600216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:14.600293 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:14.600296 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:34:14.600339 (+ 43us) client_negotiation.cc:240] Negotiation successful
0806 06:34:14.600376 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":3}
I20260806 06:34:14.600584 31539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:14.597996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:32807 (local address 127.7.43.66:52221)
0806 06:34:14.598127 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:14.598135 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:14.598190 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:14.598302 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:14.598302 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:14.598303 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:34:14.598389 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:14.598390 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:14.598966 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:14.598967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:14.599681 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:14.599683 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:14.600252 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:14.600254 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:14.600319 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:14.600322 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:34:14.600366 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 06:34:14.600402 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":75,"threads_started":1}
I20260806 06:34:14.600780 31448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e1e75cee5f14a59a4c4618b8b02164b" candidate_uuid: "f687bca681214cc79a90338958e7e828" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa4620d690aa48fd98f0d015919af71c" is_pre_election: true
I20260806 06:34:14.600911 31448 raft_consensus.cc:2454] T 3e1e75cee5f14a59a4c4618b8b02164b P fa4620d690aa48fd98f0d015919af71c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f687bca681214cc79a90338958e7e828 in term 0.
I20260806 06:34:14.600908 31024 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e1e75cee5f14a59a4c4618b8b02164b" candidate_uuid: "f687bca681214cc79a90338958e7e828" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac53c77c4c2e4eae82362a041454a59f" is_pre_election: true
I20260806 06:34:14.601042 31024 raft_consensus.cc:2454] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f687bca681214cc79a90338958e7e828 in term 0.
I20260806 06:34:14.601135 31148 leader_election.cc:302] T 3e1e75cee5f14a59a4c4618b8b02164b P f687bca681214cc79a90338958e7e828 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f687bca681214cc79a90338958e7e828, fa4620d690aa48fd98f0d015919af71c; no voters:
I20260806 06:34:14.601257 31538 raft_consensus.cc:2781] T 3e1e75cee5f14a59a4c4618b8b02164b P f687bca681214cc79a90338958e7e828 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:34:14.601323 31538 raft_consensus.cc:482] T 3e1e75cee5f14a59a4c4618b8b02164b P f687bca681214cc79a90338958e7e828 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:34:14.601349 31538 raft_consensus.cc:3037] T 3e1e75cee5f14a59a4c4618b8b02164b P f687bca681214cc79a90338958e7e828 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:34:14.601938 31538 raft_consensus.cc:504] T 3e1e75cee5f14a59a4c4618b8b02164b P f687bca681214cc79a90338958e7e828 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4620d690aa48fd98f0d015919af71c" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 46465 } } peers { permanent_uuid: "f687bca681214cc79a90338958e7e828" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 39115 } } peers { permanent_uuid: "ac53c77c4c2e4eae82362a041454a59f" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 32807 } }
I20260806 06:34:14.602047 31538 leader_election.cc:288] T 3e1e75cee5f14a59a4c4618b8b02164b P f687bca681214cc79a90338958e7e828 [CANDIDATE]: Term 1 election: Requested vote from peers fa4620d690aa48fd98f0d015919af71c (127.7.43.67:46465), ac53c77c4c2e4eae82362a041454a59f (127.7.43.65:32807)
I20260806 06:34:14.602187 31024 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e1e75cee5f14a59a4c4618b8b02164b" candidate_uuid: "f687bca681214cc79a90338958e7e828" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac53c77c4c2e4eae82362a041454a59f"
I20260806 06:34:14.602236 31448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e1e75cee5f14a59a4c4618b8b02164b" candidate_uuid: "f687bca681214cc79a90338958e7e828" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa4620d690aa48fd98f0d015919af71c"
I20260806 06:34:14.602260 31024 raft_consensus.cc:3037] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:34:14.602299 31448 raft_consensus.cc:3037] T 3e1e75cee5f14a59a4c4618b8b02164b P fa4620d690aa48fd98f0d015919af71c [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:34:14.602927 31448 raft_consensus.cc:2454] T 3e1e75cee5f14a59a4c4618b8b02164b P fa4620d690aa48fd98f0d015919af71c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f687bca681214cc79a90338958e7e828 in term 1.
I20260806 06:34:14.603085 31148 leader_election.cc:302] T 3e1e75cee5f14a59a4c4618b8b02164b P f687bca681214cc79a90338958e7e828 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f687bca681214cc79a90338958e7e828, fa4620d690aa48fd98f0d015919af71c; no voters:
I20260806 06:34:14.603184 31538 raft_consensus.cc:2781] T 3e1e75cee5f14a59a4c4618b8b02164b P f687bca681214cc79a90338958e7e828 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:34:14.603251 31024 raft_consensus.cc:2454] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f687bca681214cc79a90338958e7e828 in term 1.
I20260806 06:34:14.603323 31538 raft_consensus.cc:686] T 3e1e75cee5f14a59a4c4618b8b02164b P f687bca681214cc79a90338958e7e828 [term 1 LEADER]: Becoming Leader. State: Replica: f687bca681214cc79a90338958e7e828, State: Running, Role: LEADER
I20260806 06:34:14.603408 31538 consensus_queue.cc:237] T 3e1e75cee5f14a59a4c4618b8b02164b P f687bca681214cc79a90338958e7e828 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4620d690aa48fd98f0d015919af71c" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 46465 } } peers { permanent_uuid: "f687bca681214cc79a90338958e7e828" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 39115 } } peers { permanent_uuid: "ac53c77c4c2e4eae82362a041454a59f" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 32807 } }
I20260806 06:34:14.604205 30815 catalog_manager.cc:5495] T 3e1e75cee5f14a59a4c4618b8b02164b P f687bca681214cc79a90338958e7e828 reported cstate change: term changed from 0 to 1, leader changed from <none> to f687bca681214cc79a90338958e7e828 (127.7.43.66). New cstate: current_term: 1 leader_uuid: "f687bca681214cc79a90338958e7e828" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4620d690aa48fd98f0d015919af71c" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 46465 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f687bca681214cc79a90338958e7e828" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 39115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac53c77c4c2e4eae82362a041454a59f" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 32807 } health_report { overall_health: UNKNOWN } } }
I20260806 06:34:14.615468 31529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:14.613689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51170 (local address 127.7.43.66:39115)
0806 06:34:14.613720 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:14.613722 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:14.613731 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:34:14.613774 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:14.613774 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:14.613788 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:14.613846 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:34:14.613916 (+ 70us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:14.614317 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:14.614966 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:14.615130 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:14.615169 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:14.615261 (+ 92us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:34:14.615338 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:34:14.615386 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:14.615398 (+ 12us) server_negotiation.cc:299] Negotiation successful
0806 06:34:14.615411 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260806 06:34:14.616804 31024 raft_consensus.cc:1261] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f [term 1 FOLLOWER]: Refusing update from remote peer f687bca681214cc79a90338958e7e828: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:34:14.616807 31448 raft_consensus.cc:1261] T 3e1e75cee5f14a59a4c4618b8b02164b P fa4620d690aa48fd98f0d015919af71c [term 1 FOLLOWER]: Refusing update from remote peer f687bca681214cc79a90338958e7e828: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:34:14.617048 31538 consensus_queue.cc:1035] T 3e1e75cee5f14a59a4c4618b8b02164b P f687bca681214cc79a90338958e7e828 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac53c77c4c2e4eae82362a041454a59f" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 32807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:34:14.617172 31540 consensus_queue.cc:1035] T 3e1e75cee5f14a59a4c4618b8b02164b P f687bca681214cc79a90338958e7e828 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa4620d690aa48fd98f0d015919af71c" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 46465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:34:14.618854 31544 mvcc.cc:205] Tried to move back new op lower bound from 7315448031708491776 to 7315448031656099840. Current Snapshot: MvccSnapshot[applied={T|T < 7315448031708491776}]
I20260806 06:34:14.619060 31547 mvcc.cc:205] Tried to move back new op lower bound from 7315448031708491776 to 7315448031656099840. Current Snapshot: MvccSnapshot[applied={T|T < 7315448031708491776}]
I20260806 06:34:14.619740 31548 mvcc.cc:205] Tried to move back new op lower bound from 7315448031708491776 to 7315448031656099840. Current Snapshot: MvccSnapshot[applied={T|T < 7315448031708491776}]
I20260806 06:34:14.621404 30816 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38396:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:34:14.621475 30816 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38396:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:34:14.622066 30816 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7: Sending DeleteTablet for 3 replicas of tablet 3e1e75cee5f14a59a4c4618b8b02164b
I20260806 06:34:14.622321 30996 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e1e75cee5f14a59a4c4618b8b02164b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:34:14 UTC) from {username='kudu', principal='kudu/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:54070
I20260806 06:34:14.622433 31109 tablet_replica.cc:330] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f: stopping tablet replica
I20260806 06:34:14.622440 31421 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e1e75cee5f14a59a4c4618b8b02164b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:34:14 UTC) from {username='kudu', principal='kudu/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:33144
I20260806 06:34:14.622478 31109 raft_consensus.cc:2229] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:34:14.622522 31109 raft_consensus.cc:2258] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:34:14.622632 31224 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e1e75cee5f14a59a4c4618b8b02164b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:34:14 UTC) from {username='kudu', principal='kudu/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:51162
I20260806 06:34:14.622822 31566 tablet_replica.cc:330] T 3e1e75cee5f14a59a4c4618b8b02164b P f687bca681214cc79a90338958e7e828: stopping tablet replica
I20260806 06:34:14.622917 31566 raft_consensus.cc:2229] T 3e1e75cee5f14a59a4c4618b8b02164b P f687bca681214cc79a90338958e7e828 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:34:14.622982 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 30870
I20260806 06:34:14.623004 31566 raft_consensus.cc:2258] T 3e1e75cee5f14a59a4c4618b8b02164b P f687bca681214cc79a90338958e7e828 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:34:14.625511 31566 ts_tablet_manager.cc:1905] T 3e1e75cee5f14a59a4c4618b8b02164b P f687bca681214cc79a90338958e7e828: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:34:14.626761 31566 ts_tablet_manager.cc:1918] T 3e1e75cee5f14a59a4c4618b8b02164b P f687bca681214cc79a90338958e7e828: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:34:14.626820 31566 log.cc:1196] T 3e1e75cee5f14a59a4c4618b8b02164b P f687bca681214cc79a90338958e7e828: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-1/wal/wals/3e1e75cee5f14a59a4c4618b8b02164b
I20260806 06:34:14.627025 31566 ts_tablet_manager.cc:1939] T 3e1e75cee5f14a59a4c4618b8b02164b P f687bca681214cc79a90338958e7e828: Deleting consensus metadata
I20260806 06:34:14.627367 30795 catalog_manager.cc:4841] TS f687bca681214cc79a90338958e7e828 (127.7.43.66:39115): tablet 3e1e75cee5f14a59a4c4618b8b02164b (table test-table [id=7f2f55fc63624a16ad3c20158c06c3c9]) successfully deleted
I20260806 06:34:14.635456 31565 tablet_replica.cc:330] T 3e1e75cee5f14a59a4c4618b8b02164b P fa4620d690aa48fd98f0d015919af71c: stopping tablet replica
I20260806 06:34:14.635622 31565 raft_consensus.cc:2229] T 3e1e75cee5f14a59a4c4618b8b02164b P fa4620d690aa48fd98f0d015919af71c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:34:14.635769 31565 raft_consensus.cc:2258] T 3e1e75cee5f14a59a4c4618b8b02164b P fa4620d690aa48fd98f0d015919af71c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:34:14.636845 31565 ts_tablet_manager.cc:1905] T 3e1e75cee5f14a59a4c4618b8b02164b P fa4620d690aa48fd98f0d015919af71c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:34:14.638278 31565 ts_tablet_manager.cc:1918] T 3e1e75cee5f14a59a4c4618b8b02164b P fa4620d690aa48fd98f0d015919af71c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:34:14.638372 31565 log.cc:1196] T 3e1e75cee5f14a59a4c4618b8b02164b P fa4620d690aa48fd98f0d015919af71c: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-2/wal/wals/3e1e75cee5f14a59a4c4618b8b02164b
I20260806 06:34:14.638625 31565 ts_tablet_manager.cc:1939] T 3e1e75cee5f14a59a4c4618b8b02164b P fa4620d690aa48fd98f0d015919af71c: Deleting consensus metadata
I20260806 06:34:14.639021 30795 catalog_manager.cc:4841] TS fa4620d690aa48fd98f0d015919af71c (127.7.43.67:46465): tablet 3e1e75cee5f14a59a4c4618b8b02164b (table test-table [id=7f2f55fc63624a16ad3c20158c06c3c9]) successfully deleted
W20260806 06:34:14.641464 30793 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.43.65:32807 (error 108)
I20260806 06:34:14.641870 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 31100
I20260806 06:34:14.641996 31528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:14.641874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:32807 (local address 127.0.0.1:54080)
0806 06:34:14.641926 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:14.641963 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.65:32807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260806 06:34:14.644443 30793 catalog_manager.cc:4584] TS ac53c77c4c2e4eae82362a041454a59f (127.7.43.65:32807): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3e1e75cee5f14a59a4c4618b8b02164b: Network error: Client connection negotiation failed: client connection to 127.7.43.65:32807: connect: Connection refused (error 111)
I20260806 06:34:14.652946 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 31326
I20260806 06:34:14.662882 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 30768
I20260806 06:34:34.670429 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:33109
--webserver_interface=127.7.43.126
--webserver_port=42165
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:40943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.43.126:33109
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:34:34.750082 31569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:34:34.750286 31569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:34:34.750316 31569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:34:34.751446 31569 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 06:34:34.751485 31569 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 06:34:34.751502 31569 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 06:34:34.751518 31569 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:34:34.751533 31569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:34:34.751551 31569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:34:34.751574 31569 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:34:34.751590 31569 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:34:34.753026 31569 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:40943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.43.126:33109
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:33109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.43.126
--webserver_port=42165
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:34:34.753263 31569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:34:34.753458 31569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:34:34.758183 31579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:34:34.758196 31576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:34:34.758278 31569 server_base.cc:1034] running on GCE node
W20260806 06:34:34.758213 31575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:34:34.758711 31569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:34:34.758970 31569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:34:34.760110 31569 hybrid_clock.cc:648] HybridClock initialized: now 1785998074760096 us; error 29 us; skew 500 ppm
Aug 06 06:34:34 dist-test-slave-6zbq krb5kdc[29932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998074, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:34:34.766922 31569 init.cc:377] Logged in from keytab as kudu/127.7.43.126@KRBTEST.COM (short username kudu)
I20260806 06:34:34.767292 31569 webserver.cc:460] Webserver started at http://127.7.43.126:42165/ using document root <none> and password file <none>
I20260806 06:34:34.767457 31569 fs_manager.cc:359] Metadata directory not provided
I20260806 06:34:34.767495 31569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:34:34.768661 31569 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:34:34.769253 31591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:34.769438 31569 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:34.769519 31569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "8e48a8a8158c4faeaf4beb84cddceaa7"
format_stamp: "Formatted at 2026-08-06 06:33:53 on dist-test-slave-6zbq"
server_key: "d8709f3cd43533890902d511c9bc7330"
server_key_iv: "57e178add804559622f849671220d387"
server_key_version: "encryptionkey@0"
I20260806 06:34:34.769816 31569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:34:34.781706 31569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:34:34.783327 31569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:34:34.783425 31569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:34:34.786934 31569 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.126:33109
I20260806 06:34:34.786964 31672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.126:33109 every 8 connection(s)
I20260806 06:34:34.787279 31569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:34:34.788563 31673 sys_catalog.cc:263] Verifying existing consensus state
I20260806 06:34:34.789183 31673 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7: Bootstrap starting.
I20260806 06:34:34.795890 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 31569
I20260806 06:34:34.796243 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.65:32807
--local_ip_for_outbound_sockets=127.7.43.65
--tserver_master_addrs=127.7.43.126:33109
--webserver_port=44281
--webserver_interface=127.7.43.65
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab
--principal=kudu/127.7.43.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:40943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:34:34.797670 31673 log.cc:824] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7: Log is configured to *not* fsync() on all Append() calls
I20260806 06:34:34.799835 31673 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:34:34.800076 31673 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7: Bootstrap complete.
I20260806 06:34:34.802139 31673 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e48a8a8158c4faeaf4beb84cddceaa7" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 33109 } }
I20260806 06:34:34.802366 31673 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e48a8a8158c4faeaf4beb84cddceaa7, State: Initialized, Role: FOLLOWER
I20260806 06:34:34.802527 31673 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e48a8a8158c4faeaf4beb84cddceaa7" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 33109 } }
I20260806 06:34:34.802608 31673 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:34:34.802654 31673 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:34:34.802697 31673 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7 [term 2 FOLLOWER]: Advancing to term 3
I20260806 06:34:34.803594 31673 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e48a8a8158c4faeaf4beb84cddceaa7" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 33109 } }
I20260806 06:34:34.803687 31673 leader_election.cc:302] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e48a8a8158c4faeaf4beb84cddceaa7; no voters:
I20260806 06:34:34.803836 31673 leader_election.cc:288] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 06:34:34.803893 31677 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 06:34:34.804015 31677 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7 [term 3 LEADER]: Becoming Leader. State: Replica: 8e48a8a8158c4faeaf4beb84cddceaa7, State: Running, Role: LEADER
I20260806 06:34:34.804080 31673 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:34:34.804101 31677 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e48a8a8158c4faeaf4beb84cddceaa7" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 33109 } }
I20260806 06:34:34.804358 31680 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "8e48a8a8158c4faeaf4beb84cddceaa7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e48a8a8158c4faeaf4beb84cddceaa7" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 33109 } } }
I20260806 06:34:34.804478 31680 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7 [sys.catalog]: This master's current role is: LEADER
I20260806 06:34:34.804643 31681 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e48a8a8158c4faeaf4beb84cddceaa7. Latest consensus state: current_term: 3 leader_uuid: "8e48a8a8158c4faeaf4beb84cddceaa7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e48a8a8158c4faeaf4beb84cddceaa7" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 33109 } } }
I20260806 06:34:34.804708 31681 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7 [sys.catalog]: This master's current role is: LEADER
I20260806 06:34:34.804811 31687 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:34:34.805652 31687 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:34:34.805819 31687 catalog_manager.cc:1259] Loaded cluster ID: d3b25010ada7481c9edd95e8af43ff62
I20260806 06:34:34.805992 31687 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:34:34.806838 31687 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:34:34.807001 31687 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7: Loaded TSK: 0
I20260806 06:34:34.807039 31687 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8e48a8a8158c4faeaf4beb84cddceaa7: Loaded TSK: 1
F20260806 06:34:34.807236 31687 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fab3299c9ad google::LogMessage::Fail() at ??:0
@ 0x7fab3299f88a google::LogMessage::SendToLog() at ??:0
@ 0x7fab3299c4de google::LogMessage::Flush() at ??:0
@ 0x7fab3299da19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fab35109104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fab35106d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fab3a8f61cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fab3a8f648e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fab3a8f6594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fab3a8bcc48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fab3a8c6ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fab3486a0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fab3485f954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fab356c36db start_thread at ??:0
@ 0x7fab3149a61f clone at ??:0
W20260806 06:34:34.894064 31676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:34:34.894249 31676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:34:34.894271 31676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:34:34.895414 31676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:34:34.895454 31676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:34:34.895479 31676 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.65
I20260806 06:34:34.896883 31676 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:40943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab
--principal=kudu/127.7.43.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.65:32807
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.43.65
--webserver_port=44281
--tserver_master_addrs=127.7.43.126:33109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31676
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.65
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:34:34.897090 31676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:34:34.897312 31676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:34:34.902066 31737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:34:34.902066 31732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:34:34.902066 31731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:34:34.902398 31676 server_base.cc:1034] running on GCE node
I20260806 06:34:34.902580 31676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:34:34.902804 31676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:34:34.903959 31676 hybrid_clock.cc:648] HybridClock initialized: now 1785998074903948 us; error 26 us; skew 500 ppm
Aug 06 06:34:34 dist-test-slave-6zbq krb5kdc[29932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998074, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:34:34.906445 31676 init.cc:377] Logged in from keytab as kudu/127.7.43.65@KRBTEST.COM (short username kudu)
I20260806 06:34:34.906852 31676 webserver.cc:460] Webserver started at http://127.7.43.65:44281/ using document root <none> and password file <none>
I20260806 06:34:34.907068 31676 fs_manager.cc:359] Metadata directory not provided
I20260806 06:34:34.907115 31676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:34:34.908411 31676 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:34:34.909067 31753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:34.909248 31676 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:34:34.909343 31676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-0/wal
uuid: "ac53c77c4c2e4eae82362a041454a59f"
format_stamp: "Formatted at 2026-08-06 06:33:53 on dist-test-slave-6zbq"
server_key: "d0002476731721625c938e16e5d01a0c"
server_key_iv: "71847b749b16484bbcda29cb524aaf35"
server_key_version: "encryptionkey@0"
I20260806 06:34:34.909730 31676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:34:34.931501 31676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:34:34.933516 31676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:34:34.933619 31676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:34:34.933832 31676 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:34:34.934228 31764 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 06:34:34.936136 31676 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 06:34:34.936177 31676 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 06:34:34.936208 31676 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 06:34:34.936812 31676 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 06:34:34.936844 31676 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 06:34:34.936935 31764 tablet_bootstrap.cc:492] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f: Bootstrap starting.
I20260806 06:34:34.943343 31676 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.65:32807
I20260806 06:34:34.943497 31928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.65:32807 every 8 connection(s)
I20260806 06:34:34.943732 31676 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
I20260806 06:34:34.947831 31764 log.cc:824] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f: Log is configured to *not* fsync() on all Append() calls
I20260806 06:34:34.950382 31764 tablet_bootstrap.cc:492] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:34:34.950697 31764 tablet_bootstrap.cc:492] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f: Bootstrap complete.
I20260806 06:34:34.951820 31764 ts_tablet_manager.cc:1397] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.007s
I20260806 06:34:34.952298 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 31676
I20260806 06:34:34.952550 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.66:39115
--local_ip_for_outbound_sockets=127.7.43.66
--tserver_master_addrs=127.7.43.126:33109
--webserver_port=36801
--webserver_interface=127.7.43.66
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab
--principal=kudu/127.7.43.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:40943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:34:34.953147 31764 raft_consensus.cc:348] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4620d690aa48fd98f0d015919af71c" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 46465 } } peers { permanent_uuid: "f687bca681214cc79a90338958e7e828" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 39115 } } peers { permanent_uuid: "ac53c77c4c2e4eae82362a041454a59f" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 32807 } }
I20260806 06:34:34.953450 31764 raft_consensus.cc:729] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac53c77c4c2e4eae82362a041454a59f, State: Initialized, Role: FOLLOWER
I20260806 06:34:34.953608 31764 consensus_queue.cc:260] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4620d690aa48fd98f0d015919af71c" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 46465 } } peers { permanent_uuid: "f687bca681214cc79a90338958e7e828" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 39115 } } peers { permanent_uuid: "ac53c77c4c2e4eae82362a041454a59f" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 32807 } }
I20260806 06:34:34.953853 31764 ts_tablet_manager.cc:1428] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f: Time spent starting tablet: real 0.002s user 0.003s sys 0.001s
W20260806 06:34:34.996225 31931 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:34:34.944296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.65:60047)
0806 06:34:34.946532 (+ 2236us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:34.946563 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:34.946966 (+ 403us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:34.996033 (+ 49067us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:33109: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":2205,"thread_start_us":69,"threads_started":1}
W20260806 06:34:34.996518 31929 heartbeater.cc:644] Failed to heartbeat to 127.7.43.126:33109 (0 consecutive failures): Network error: Failed to ping master at 127.7.43.126:33109: Client connection negotiation failed: client connection to 127.7.43.126:33109: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 06:34:34.996896 31931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:34.996758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.65:33617)
0806 06:34:34.996817 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:34.996868 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 06:34:34.997342 31931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:34.997235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.65:60287)
0806 06:34:34.997293 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:34.997321 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260806 06:34:34.997471 31929 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 06:34:35.034511 31937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:34:35.034677 31937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:34:35.034698 31937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:34:35.035806 31937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:34:35.035846 31937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:34:35.035874 31937 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.66
I20260806 06:34:35.037274 31937 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:40943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab
--principal=kudu/127.7.43.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.66:39115
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.43.66
--webserver_port=36801
--tserver_master_addrs=127.7.43.126:33109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31937
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.66
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:34:35.037480 31937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:34:35.037688 31937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:34:35.042433 31945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:34:35.042390 31949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:34:35.042548 31937 server_base.cc:1034] running on GCE node
W20260806 06:34:35.042393 31944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:34:35.042821 31937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:34:35.043047 31937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:34:35.044176 31937 hybrid_clock.cc:648] HybridClock initialized: now 1785998075044157 us; error 36 us; skew 500 ppm
Aug 06 06:34:35 dist-test-slave-6zbq krb5kdc[29932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998075, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:34:35.046505 31937 init.cc:377] Logged in from keytab as kudu/127.7.43.66@KRBTEST.COM (short username kudu)
I20260806 06:34:35.046892 31937 webserver.cc:460] Webserver started at http://127.7.43.66:36801/ using document root <none> and password file <none>
I20260806 06:34:35.047102 31937 fs_manager.cc:359] Metadata directory not provided
I20260806 06:34:35.047145 31937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:34:35.048316 31937 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:34:35.048821 31959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:35.048983 31937 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:34:35.049062 31937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-1/wal
uuid: "f687bca681214cc79a90338958e7e828"
format_stamp: "Formatted at 2026-08-06 06:33:53 on dist-test-slave-6zbq"
server_key: "272b00e53b95fcb22e21f39eae054230"
server_key_iv: "2f8559e478d2a401bd2c11b9c9cbf464"
server_key_version: "encryptionkey@0"
I20260806 06:34:35.049352 31937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:34:35.064543 31937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:34:35.066166 31937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:34:35.066298 31937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:34:35.066517 31937 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:34:35.066778 31937 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:34:35.066811 31937 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:35.066838 31937 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:34:35.066852 31937 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:35.072075 31937 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.66:39115
I20260806 06:34:35.072109 32126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.66:39115 every 8 connection(s)
I20260806 06:34:35.072408 31937 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
I20260806 06:34:35.073307 32130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:35.072983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.66:49977)
0806 06:34:35.073117 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:35.073177 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":66,"threads_started":1}
W20260806 06:34:35.073652 32128 heartbeater.cc:644] Failed to heartbeat to 127.7.43.126:33109 (0 consecutive failures): Network error: Failed to ping master at 127.7.43.126:33109: Client connection negotiation failed: client connection to 127.7.43.126:33109: connect: Connection refused (error 111)
I20260806 06:34:35.073886 32130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:35.073803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.66:35313)
0806 06:34:35.073845 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:35.073870 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 06:34:35.074100 32130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:35.074037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.66:35121)
0806 06:34:35.074070 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:35.074087 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260806 06:34:35.074191 32128 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 06:34:35.078143 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 31937
I20260806 06:34:35.078428 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.67:46465
--local_ip_for_outbound_sockets=127.7.43.67
--tserver_master_addrs=127.7.43.126:33109
--webserver_port=46467
--webserver_interface=127.7.43.67
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab
--principal=kudu/127.7.43.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:40943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:34:35.156824 32131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:34:35.156991 32131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:34:35.157012 32131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:34:35.158056 32131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:34:35.158087 32131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:34:35.158111 32131 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.67
I20260806 06:34:35.159564 32131 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:40943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab
--principal=kudu/127.7.43.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.67:46465
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.43.67
--webserver_port=46467
--tserver_master_addrs=127.7.43.126:33109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32131
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.67
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:34:35.159757 32131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:34:35.159922 32131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:34:35.164464 32137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:34:35.164552 32144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:34:35.164481 32139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:34:35.164597 32131 server_base.cc:1034] running on GCE node
I20260806 06:34:35.164827 32131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:34:35.165040 32131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:34:35.166179 32131 hybrid_clock.cc:648] HybridClock initialized: now 1785998075166161 us; error 29 us; skew 500 ppm
Aug 06 06:34:35 dist-test-slave-6zbq krb5kdc[29932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998075, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:34:35.168432 32131 init.cc:377] Logged in from keytab as kudu/127.7.43.67@KRBTEST.COM (short username kudu)
I20260806 06:34:35.168834 32131 webserver.cc:460] Webserver started at http://127.7.43.67:46467/ using document root <none> and password file <none>
I20260806 06:34:35.169035 32131 fs_manager.cc:359] Metadata directory not provided
I20260806 06:34:35.169081 32131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:34:35.170243 32131 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:34:35.170776 32156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:35.170926 32131 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:35.170995 32131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-2/wal
uuid: "fa4620d690aa48fd98f0d015919af71c"
format_stamp: "Formatted at 2026-08-06 06:33:53 on dist-test-slave-6zbq"
server_key: "2d4fb9cda3831e3ce0b052e141b01db3"
server_key_iv: "7ccd7702495d8a94d4b0ed26e1291135"
server_key_version: "encryptionkey@0"
I20260806 06:34:35.171268 32131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:34:35.188315 32131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:34:35.190208 32131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:34:35.190335 32131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:34:35.190568 32131 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:34:35.190842 32131 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:34:35.190874 32131 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:35.190900 32131 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:34:35.190917 32131 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:35.196386 32131 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.67:46465
I20260806 06:34:35.196422 32316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.67:46465 every 8 connection(s)
I20260806 06:34:35.196723 32131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
I20260806 06:34:35.197607 32324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:35.197283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.67:39241)
0806 06:34:35.197422 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:35.197481 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":70,"threads_started":1}
W20260806 06:34:35.197964 32317 heartbeater.cc:644] Failed to heartbeat to 127.7.43.126:33109 (0 consecutive failures): Network error: Failed to ping master at 127.7.43.126:33109: Client connection negotiation failed: client connection to 127.7.43.126:33109: connect: Connection refused (error 111)
I20260806 06:34:35.198196 32324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:35.198114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.67:57709)
0806 06:34:35.198156 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:35.198180 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 06:34:35.198436 32324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:35.198384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.67:37803)
0806 06:34:35.198406 (+ 22us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:35.198424 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260806 06:34:35.198526 32317 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 06:34:35.203567 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 32131
I20260806 06:34:35.997988 32327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:35.997780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.65:51461)
0806 06:34:35.997895 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:35.997929 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":57,"threads_started":1}
I20260806 06:34:36.074671 32328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:36.074481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.66:34101)
0806 06:34:36.074589 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:36.074623 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":54,"threads_started":1}
I20260806 06:34:36.198971 32329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:36.198785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.67:33921)
0806 06:34:36.198892 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:36.198923 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":53,"threads_started":1}
I20260806 06:34:36.291819 32330 raft_consensus.cc:482] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:34:36.291937 32330 raft_consensus.cc:504] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4620d690aa48fd98f0d015919af71c" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 46465 } } peers { permanent_uuid: "f687bca681214cc79a90338958e7e828" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 39115 } } peers { permanent_uuid: "ac53c77c4c2e4eae82362a041454a59f" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 32807 } }
I20260806 06:34:36.292258 32330 leader_election.cc:288] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fa4620d690aa48fd98f0d015919af71c (127.7.43.67:46465), f687bca681214cc79a90338958e7e828 (127.7.43.66:39115)
Aug 06 06:34:36 dist-test-slave-6zbq krb5kdc[29932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998074, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.65@KRBTEST.COM for kudu/127.7.43.67@KRBTEST.COM
Aug 06 06:34:36 dist-test-slave-6zbq krb5kdc[29932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998074, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.65@KRBTEST.COM for kudu/127.7.43.66@KRBTEST.COM
I20260806 06:34:36.301913 32331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:36.292494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.66:39115 (local address 127.7.43.65:54345)
0806 06:34:36.292635 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:36.292644 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:36.292700 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:36.293417 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:36.293420 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:36.293808 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:34:36.293997 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:36.294003 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:36.295102 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:36.295103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:36.295856 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:36.295858 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:36.296415 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:36.296416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:36.296530 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:36.297039 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:34:36.297048 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:34:36.298809 (+ 1761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:34:36.300518 (+ 1709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:34:36.300519 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:34:36.300520 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:34:36.300678 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:34:36.300909 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:34:36.300910 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:34:36.300910 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:34:36.300968 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:34:36.301421 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:34:36.301422 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:34:36.301521 (+ 99us) client_negotiation.cc:769] Sending connection context
0806 06:34:36.301629 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 06:34:36.301805 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":92,"thread_start_us":74,"threads_started":1}
I20260806 06:34:36.301947 32332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:36.292527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:44615 (local address 127.7.43.67:46465)
0806 06:34:36.292661 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:36.292673 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:36.292709 (+ 36us) server_negotiation.cc:407] Connection header received
0806 06:34:36.293165 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:36.293167 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:36.293209 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:36.293350 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:34:36.294116 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:36.295002 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:36.295941 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:36.296326 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:36.296422 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:36.298545 (+ 2123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:34:36.298548 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:34:36.298549 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:34:36.298559 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:34:36.300235 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:34:36.300611 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:34:36.300614 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:34:36.300614 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:34:36.300703 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:34:36.300936 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:34:36.300937 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:34:36.300937 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:34:36.301251 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:34:36.301338 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:36.301697 (+ 359us) server_negotiation.cc:299] Negotiation successful
0806 06:34:36.301833 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":59,"threads_started":1}
I20260806 06:34:36.302107 32327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:36.292387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.67:46465 (local address 127.7.43.65:44615)
0806 06:34:36.292443 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:36.292455 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:36.292529 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:36.293416 (+ 887us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:36.293420 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:36.293800 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:34:36.293997 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:36.294003 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:36.295105 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:36.295107 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:36.295854 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:36.295857 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:36.296413 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:36.296414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:36.296530 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:36.297089 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:34:36.297094 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:34:36.298448 (+ 1354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:34:36.300317 (+ 1869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:34:36.300324 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:34:36.300326 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:34:36.300529 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:34:36.300770 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:34:36.300771 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:34:36.300772 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:34:36.300873 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:34:36.301372 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:34:36.301379 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:34:36.301521 (+ 142us) client_negotiation.cc:769] Sending connection context
0806 06:34:36.301629 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 06:34:36.301876 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":116}
I20260806 06:34:36.301944 32333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:36.292613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:54345 (local address 127.7.43.66:39115)
0806 06:34:36.292781 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:36.292790 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:36.292816 (+ 26us) server_negotiation.cc:407] Connection header received
0806 06:34:36.293237 (+ 421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:36.293239 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:36.293278 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:36.293396 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:34:36.294116 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:36.295001 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:36.295945 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:36.296327 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:36.296422 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:36.298913 (+ 2491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:34:36.298917 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:34:36.298918 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:34:36.298927 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:34:36.300438 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:34:36.300753 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:34:36.300756 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:34:36.300756 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:34:36.300841 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:34:36.301028 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:34:36.301029 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:34:36.301029 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:34:36.301317 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:34:36.301396 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:36.301698 (+ 302us) server_negotiation.cc:299] Negotiation successful
0806 06:34:36.301833 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":69,"threads_started":1}
W20260806 06:34:36.302842 31757 leader_election.cc:341] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fa4620d690aa48fd98f0d015919af71c (127.7.43.67:46465): Not found: Tablet not found: 3e1e75cee5f14a59a4c4618b8b02164b
W20260806 06:34:36.302917 31757 leader_election.cc:341] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f687bca681214cc79a90338958e7e828 (127.7.43.66:39115): Not found: Tablet not found: 3e1e75cee5f14a59a4c4618b8b02164b
I20260806 06:34:36.302939 31757 leader_election.cc:302] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ac53c77c4c2e4eae82362a041454a59f; no voters: f687bca681214cc79a90338958e7e828, fa4620d690aa48fd98f0d015919af71c
I20260806 06:34:36.303026 32330 raft_consensus.cc:2726] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 06:34:36.998646 32335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:36.998445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.65:45687)
0806 06:34:36.998567 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:36.998601 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":58,"threads_started":1}
I20260806 06:34:37.075282 32336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:37.075080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.66:40557)
0806 06:34:37.075209 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:37.075249 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":79,"threads_started":1}
I20260806 06:34:37.199540 32337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:37.199364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.67:45785)
0806 06:34:37.199470 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:37.199503 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":57,"threads_started":1}
I20260806 06:34:37.999250 32339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:37.999053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.65:40747)
0806 06:34:37.999175 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:37.999207 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":71,"threads_started":1}
I20260806 06:34:38.075850 32340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:38.075654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.66:36437)
0806 06:34:38.075779 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:38.075814 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":55,"threads_started":1}
I20260806 06:34:38.200104 32342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:38.199919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.67:43011)
0806 06:34:38.200032 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:38.200067 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":61,"threads_started":1}
I20260806 06:34:38.472873 32343 raft_consensus.cc:482] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:34:38.472965 32343 raft_consensus.cc:504] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4620d690aa48fd98f0d015919af71c" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 46465 } } peers { permanent_uuid: "f687bca681214cc79a90338958e7e828" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 39115 } } peers { permanent_uuid: "ac53c77c4c2e4eae82362a041454a59f" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 32807 } }
I20260806 06:34:38.473268 32343 leader_election.cc:288] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fa4620d690aa48fd98f0d015919af71c (127.7.43.67:46465), f687bca681214cc79a90338958e7e828 (127.7.43.66:39115)
W20260806 06:34:38.473572 31757 leader_election.cc:341] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f687bca681214cc79a90338958e7e828 (127.7.43.66:39115): Not found: Tablet not found: 3e1e75cee5f14a59a4c4618b8b02164b
W20260806 06:34:38.473647 31757 leader_election.cc:341] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fa4620d690aa48fd98f0d015919af71c (127.7.43.67:46465): Not found: Tablet not found: 3e1e75cee5f14a59a4c4618b8b02164b
I20260806 06:34:38.473670 31757 leader_election.cc:302] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ac53c77c4c2e4eae82362a041454a59f; no voters: f687bca681214cc79a90338958e7e828, fa4620d690aa48fd98f0d015919af71c
I20260806 06:34:38.473785 32343 raft_consensus.cc:2726] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 06:34:38.999828 32346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:38.999617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.65:39035)
0806 06:34:38.999749 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:38.999784 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":53,"threads_started":1}
I20260806 06:34:39.076424 32347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:39.076246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.66:38323)
0806 06:34:39.076355 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:39.076389 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":56,"threads_started":1}
I20260806 06:34:39.200713 32350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:39.200533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.67:48065)
0806 06:34:39.200644 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:39.200676 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":61,"threads_started":1}
I20260806 06:34:40.000478 32351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:40.000261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.65:57537)
0806 06:34:40.000397 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:40.000432 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":57,"threads_started":1}
I20260806 06:34:40.076975 32352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:40.076768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.66:46729)
0806 06:34:40.076907 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:40.076939 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":69,"threads_started":1}
I20260806 06:34:40.201277 32353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:40.201055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.67:33781)
0806 06:34:40.201208 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:40.201240 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":78,"threads_started":1}
I20260806 06:34:40.820466 32355 raft_consensus.cc:482] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:34:40.820555 32355 raft_consensus.cc:504] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4620d690aa48fd98f0d015919af71c" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 46465 } } peers { permanent_uuid: "f687bca681214cc79a90338958e7e828" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 39115 } } peers { permanent_uuid: "ac53c77c4c2e4eae82362a041454a59f" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 32807 } }
I20260806 06:34:40.820721 32355 leader_election.cc:288] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fa4620d690aa48fd98f0d015919af71c (127.7.43.67:46465), f687bca681214cc79a90338958e7e828 (127.7.43.66:39115)
W20260806 06:34:40.821062 31757 leader_election.cc:341] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fa4620d690aa48fd98f0d015919af71c (127.7.43.67:46465): Not found: Tablet not found: 3e1e75cee5f14a59a4c4618b8b02164b
W20260806 06:34:40.821167 31757 leader_election.cc:341] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f687bca681214cc79a90338958e7e828 (127.7.43.66:39115): Not found: Tablet not found: 3e1e75cee5f14a59a4c4618b8b02164b
I20260806 06:34:40.821200 31757 leader_election.cc:302] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ac53c77c4c2e4eae82362a041454a59f; no voters: f687bca681214cc79a90338958e7e828, fa4620d690aa48fd98f0d015919af71c
I20260806 06:34:40.821292 32355 raft_consensus.cc:2726] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 06:34:41.001060 32357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:41.000869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.65:48505)
0806 06:34:41.000984 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:41.001016 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":57,"threads_started":1}
I20260806 06:34:41.077551 32359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:41.077358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.66:34663)
0806 06:34:41.077484 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:41.077516 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":66,"threads_started":1}
I20260806 06:34:41.201851 32361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:41.201642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.67:42937)
0806 06:34:41.201778 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:41.201813 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":71,"threads_started":1}
I20260806 06:34:42.001657 32362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:42.001462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.65:34215)
0806 06:34:42.001577 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:42.001611 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":60,"threads_started":1}
I20260806 06:34:42.078102 32363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:42.077920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.66:49831)
0806 06:34:42.078032 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:42.078066 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":52,"threads_started":1}
I20260806 06:34:42.202468 32364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:42.202253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.67:49411)
0806 06:34:42.202401 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:42.202432 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":59,"threads_started":1}
I20260806 06:34:43.002245 32369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:43.002022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.65:43631)
0806 06:34:43.002167 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:43.002201 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":59,"threads_started":1}
I20260806 06:34:43.078890 32371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:43.078708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.66:34573)
0806 06:34:43.078819 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:43.078852 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":58,"threads_started":1}
I20260806 06:34:43.203083 32372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:43.202872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.67:54669)
0806 06:34:43.203012 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:43.203045 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":71,"threads_started":1}
I20260806 06:34:44.002898 32373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:44.002674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.65:34997)
0806 06:34:44.002814 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:44.002852 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":60,"threads_started":1}
I20260806 06:34:44.079463 32375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:44.079240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.66:41029)
0806 06:34:44.079373 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:44.079414 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":76,"threads_started":1}
I20260806 06:34:44.195851 32377 raft_consensus.cc:482] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:34:44.195947 32377 raft_consensus.cc:504] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4620d690aa48fd98f0d015919af71c" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 46465 } } peers { permanent_uuid: "f687bca681214cc79a90338958e7e828" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 39115 } } peers { permanent_uuid: "ac53c77c4c2e4eae82362a041454a59f" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 32807 } }
I20260806 06:34:44.196103 32377 leader_election.cc:288] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fa4620d690aa48fd98f0d015919af71c (127.7.43.67:46465), f687bca681214cc79a90338958e7e828 (127.7.43.66:39115)
W20260806 06:34:44.196524 31757 leader_election.cc:341] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f687bca681214cc79a90338958e7e828 (127.7.43.66:39115): Not found: Tablet not found: 3e1e75cee5f14a59a4c4618b8b02164b
W20260806 06:34:44.196597 31757 leader_election.cc:341] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fa4620d690aa48fd98f0d015919af71c (127.7.43.67:46465): Not found: Tablet not found: 3e1e75cee5f14a59a4c4618b8b02164b
I20260806 06:34:44.196620 31757 leader_election.cc:302] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ac53c77c4c2e4eae82362a041454a59f; no voters: f687bca681214cc79a90338958e7e828, fa4620d690aa48fd98f0d015919af71c
I20260806 06:34:44.196707 32377 raft_consensus.cc:2726] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 06:34:44.203742 32379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:44.203539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.67:35947)
0806 06:34:44.203661 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:44.203700 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":60,"threads_started":1}
I20260806 06:34:45.003497 32381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:45.003305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.65:37883)
0806 06:34:45.003414 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:45.003447 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":60,"threads_started":1}
I20260806 06:34:45.080040 32383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:45.079856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.66:48893)
0806 06:34:45.079969 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:45.080002 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":56,"threads_started":1}
I20260806 06:34:45.204335 32386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:45.204118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.67:43467)
0806 06:34:45.204263 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:45.204298 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":62,"threads_started":1}
I20260806 06:34:46.004136 32389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:46.003898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.65:40055)
0806 06:34:46.004053 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:46.004090 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":79,"threads_started":1}
I20260806 06:34:46.080650 32390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:46.080433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.66:48521)
0806 06:34:46.080574 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:46.080610 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":78,"threads_started":1}
I20260806 06:34:46.204958 32392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:46.204765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.67:52065)
0806 06:34:46.204883 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:46.204918 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":61,"threads_started":1}
I20260806 06:34:47.004789 32393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:47.004591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.65:38817)
0806 06:34:47.004701 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:47.004743 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":59,"threads_started":1}
I20260806 06:34:47.081272 32395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:47.081068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.66:46751)
0806 06:34:47.081199 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:47.081232 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":78,"threads_started":1}
I20260806 06:34:47.102236 32396 raft_consensus.cc:482] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:34:47.102315 32396 raft_consensus.cc:504] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4620d690aa48fd98f0d015919af71c" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 46465 } } peers { permanent_uuid: "f687bca681214cc79a90338958e7e828" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 39115 } } peers { permanent_uuid: "ac53c77c4c2e4eae82362a041454a59f" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 32807 } }
I20260806 06:34:47.102475 32396 leader_election.cc:288] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fa4620d690aa48fd98f0d015919af71c (127.7.43.67:46465), f687bca681214cc79a90338958e7e828 (127.7.43.66:39115)
W20260806 06:34:47.102813 31757 leader_election.cc:341] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f687bca681214cc79a90338958e7e828 (127.7.43.66:39115): Not found: Tablet not found: 3e1e75cee5f14a59a4c4618b8b02164b
W20260806 06:34:47.102876 31757 leader_election.cc:341] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fa4620d690aa48fd98f0d015919af71c (127.7.43.67:46465): Not found: Tablet not found: 3e1e75cee5f14a59a4c4618b8b02164b
I20260806 06:34:47.102893 31757 leader_election.cc:302] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ac53c77c4c2e4eae82362a041454a59f; no voters: f687bca681214cc79a90338958e7e828, fa4620d690aa48fd98f0d015919af71c
I20260806 06:34:47.102964 32396 raft_consensus.cc:2726] T 3e1e75cee5f14a59a4c4618b8b02164b P ac53c77c4c2e4eae82362a041454a59f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 06:34:47.205535 32399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:47.205332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.67:35923)
0806 06:34:47.205456 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:47.205495 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":58,"threads_started":1}
I20260806 06:34:48.005381 32400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:48.005193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.65:51715)
0806 06:34:48.005303 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:48.005336 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":60,"threads_started":1}
I20260806 06:34:48.081812 32403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:48.081617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.66:56427)
0806 06:34:48.081733 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:48.081766 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":68,"threads_started":1}
I20260806 06:34:48.206079 32404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:48.205902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.67:59071)
0806 06:34:48.206011 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:48.206045 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":56,"threads_started":1}
I20260806 06:34:49.005944 32405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:49.005751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.65:55639)
0806 06:34:49.005865 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:49.005897 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":57,"threads_started":1}
I20260806 06:34:49.082399 32407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:49.082161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.66:46321)
0806 06:34:49.082300 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:49.082359 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":80,"threads_started":1}
I20260806 06:34:49.206645 32408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:49.206450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.67:45535)
0806 06:34:49.206564 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:49.206598 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":58,"threads_started":1}
I20260806 06:34:50.006577 32410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:50.006383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.65:55951)
0806 06:34:50.006498 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:50.006531 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":57,"threads_started":1}
I20260806 06:34:50.083021 32411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:50.082836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.66:38815)
0806 06:34:50.082953 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:50.082982 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":71,"threads_started":1}
I20260806 06:34:50.204123 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 31676
I20260806 06:34:50.208644 32413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:50.208439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:33109 (local address 127.7.43.67:56653)
0806 06:34:50.208566 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:50.208596 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.126:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":55,"threads_started":1}
I20260806 06:34:50.225112 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 31937
I20260806 06:34:50.234793 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 32131
2026-08-06T06:34:50Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58156 ms)
[----------] 30 tests from SecurityITest (228403 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:34:50 dist-test-slave-6zbq krb5kdc[32418](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:34:50 dist-test-slave-6zbq krb5kdc[32418](info): set up 2 sockets
Aug 06 06:34:50 dist-test-slave-6zbq krb5kdc[32418](info): commencing operation
krb5kdc: starting...
W20260806 06:34:51.329139 7341 mini_kdc.cc:121] Time spent starting KDC: real 1.072s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Aug 06 06:34:51 dist-test-slave-6zbq krb5kdc[32418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998091, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:34:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:34:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.43.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
I20260806 06:34:51.434754 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:40111
--webserver_interface=127.7.43.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:36017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.43.126:40111
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:34:51.515403 32438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:34:51.515578 32438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:34:51.515606 32438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:34:51.516685 32438 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:34:51.516724 32438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:34:51.516743 32438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:34:51.516759 32438 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:34:51.516785 32438 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:34:51.518186 32438 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:36017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.43.126:40111
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:40111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.43.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32438
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:34:51.518431 32438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:34:51.518615 32438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:34:51.523016 32445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:34:51.523026 32448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:34:51.523001 32446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:34:51.523157 32438 server_base.cc:1034] running on GCE node
I20260806 06:34:51.523499 32438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:34:51.523749 32438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:34:51.524881 32438 hybrid_clock.cc:648] HybridClock initialized: now 1785998091524869 us; error 29 us; skew 500 ppm
Aug 06 06:34:51 dist-test-slave-6zbq krb5kdc[32418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998091, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:34:51.527311 32438 init.cc:377] Logged in from keytab as kudu/127.7.43.126@KRBTEST.COM (short username kudu)
I20260806 06:34:51.527724 32438 webserver.cc:460] Webserver started at http://127.7.43.126:39833/ using document root <none> and password file <none>
I20260806 06:34:51.527905 32438 fs_manager.cc:359] Metadata directory not provided
I20260806 06:34:51.527946 32438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:34:51.528076 32438 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:34:51.529093 32438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/master-0/data/instance:
uuid: "e83f6d77083d462cab65c301161189d1"
format_stamp: "Formatted at 2026-08-06 06:34:51 on dist-test-slave-6zbq"
server_key: "c065182022955f5597e2543b3e700178"
server_key_iv: "81aae10ee5d662ce75b7249740c60c23"
server_key_version: "encryptionkey@0"
I20260806 06:34:51.529480 32438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/master-0/wal/instance:
uuid: "e83f6d77083d462cab65c301161189d1"
format_stamp: "Formatted at 2026-08-06 06:34:51 on dist-test-slave-6zbq"
server_key: "c065182022955f5597e2543b3e700178"
server_key_iv: "81aae10ee5d662ce75b7249740c60c23"
server_key_version: "encryptionkey@0"
I20260806 06:34:51.530880 32438 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:34:51.531504 32456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:51.531642 32438 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:34:51.531725 32438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "e83f6d77083d462cab65c301161189d1"
format_stamp: "Formatted at 2026-08-06 06:34:51 on dist-test-slave-6zbq"
server_key: "c065182022955f5597e2543b3e700178"
server_key_iv: "81aae10ee5d662ce75b7249740c60c23"
server_key_version: "encryptionkey@0"
I20260806 06:34:51.531785 32438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:34:51.568271 32438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:34:51.569831 32438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:34:51.569954 32438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:34:51.573565 32438 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.126:40111
I20260806 06:34:51.573603 32533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.126:40111 every 8 connection(s)
I20260806 06:34:51.573890 32438 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:34:51.574666 32534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:34:51.576704 32534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e83f6d77083d462cab65c301161189d1: Bootstrap starting.
I20260806 06:34:51.577201 32534 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e83f6d77083d462cab65c301161189d1: Neither blocks nor log segments found. Creating new log.
I20260806 06:34:51.577425 32534 log.cc:824] T 00000000000000000000000000000000 P e83f6d77083d462cab65c301161189d1: Log is configured to *not* fsync() on all Append() calls
I20260806 06:34:51.578011 32534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e83f6d77083d462cab65c301161189d1: No bootstrap required, opened a new log
I20260806 06:34:51.578725 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 32438
I20260806 06:34:51.578828 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/master-0/wal/instance
I20260806 06:34:51.578999 7341 external_mini_cluster.cc:1428] Setting key ea4f320a08bf757fbdc87e11145a2b52
I20260806 06:34:51.579905 32534 raft_consensus.cc:348] T 00000000000000000000000000000000 P e83f6d77083d462cab65c301161189d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e83f6d77083d462cab65c301161189d1" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 40111 } }
I20260806 06:34:51.580065 32534 raft_consensus.cc:374] T 00000000000000000000000000000000 P e83f6d77083d462cab65c301161189d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:34:51.580138 32534 raft_consensus.cc:729] T 00000000000000000000000000000000 P e83f6d77083d462cab65c301161189d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e83f6d77083d462cab65c301161189d1, State: Initialized, Role: FOLLOWER
I20260806 06:34:51.580267 32534 consensus_queue.cc:260] T 00000000000000000000000000000000 P e83f6d77083d462cab65c301161189d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e83f6d77083d462cab65c301161189d1" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 40111 } }
I20260806 06:34:51.580343 32534 raft_consensus.cc:388] T 00000000000000000000000000000000 P e83f6d77083d462cab65c301161189d1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:34:51.580375 32534 raft_consensus.cc:482] T 00000000000000000000000000000000 P e83f6d77083d462cab65c301161189d1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:34:51.580420 32534 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e83f6d77083d462cab65c301161189d1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:34:51.581504 32534 raft_consensus.cc:504] T 00000000000000000000000000000000 P e83f6d77083d462cab65c301161189d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e83f6d77083d462cab65c301161189d1" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 40111 } }
I20260806 06:34:51.581609 32534 leader_election.cc:302] T 00000000000000000000000000000000 P e83f6d77083d462cab65c301161189d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e83f6d77083d462cab65c301161189d1; no voters:
I20260806 06:34:51.581755 32534 leader_election.cc:288] T 00000000000000000000000000000000 P e83f6d77083d462cab65c301161189d1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:34:51.581784 32543 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e83f6d77083d462cab65c301161189d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:34:51.581946 32543 raft_consensus.cc:686] T 00000000000000000000000000000000 P e83f6d77083d462cab65c301161189d1 [term 1 LEADER]: Becoming Leader. State: Replica: e83f6d77083d462cab65c301161189d1, State: Running, Role: LEADER
I20260806 06:34:51.581966 32534 sys_catalog.cc:564] T 00000000000000000000000000000000 P e83f6d77083d462cab65c301161189d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:34:51.582046 32543 consensus_queue.cc:237] T 00000000000000000000000000000000 P e83f6d77083d462cab65c301161189d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e83f6d77083d462cab65c301161189d1" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 40111 } }
I20260806 06:34:51.582420 32543 sys_catalog.cc:455] T 00000000000000000000000000000000 P e83f6d77083d462cab65c301161189d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e83f6d77083d462cab65c301161189d1. Latest consensus state: current_term: 1 leader_uuid: "e83f6d77083d462cab65c301161189d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e83f6d77083d462cab65c301161189d1" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 40111 } } }
I20260806 06:34:51.582432 32544 sys_catalog.cc:455] T 00000000000000000000000000000000 P e83f6d77083d462cab65c301161189d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e83f6d77083d462cab65c301161189d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e83f6d77083d462cab65c301161189d1" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 40111 } } }
I20260806 06:34:51.582576 32543 sys_catalog.cc:458] T 00000000000000000000000000000000 P e83f6d77083d462cab65c301161189d1 [sys.catalog]: This master's current role is: LEADER
I20260806 06:34:51.582583 32544 sys_catalog.cc:458] T 00000000000000000000000000000000 P e83f6d77083d462cab65c301161189d1 [sys.catalog]: This master's current role is: LEADER
I20260806 06:34:51.582789 32558 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:34:51.583444 32558 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:34:51.585130 32558 catalog_manager.cc:1347] Generated new cluster ID: b4d1fc554c944487858c46dc2e5aa6be
I20260806 06:34:51.585183 32558 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:34:51 dist-test-slave-6zbq krb5kdc[32418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998091, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:34:51.588907 32541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:51.579446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44826 (local address 127.7.43.126:40111)
0806 06:34:51.579628 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:51.579638 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:51.579670 (+ 32us) server_negotiation.cc:407] Connection header received
0806 06:34:51.579864 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:51.579872 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:51.579927 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:51.580069 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:34:51.580732 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:51.582052 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:51.583169 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:51.583731 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:51.583831 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:51.586178 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:34:51.586182 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:34:51.586182 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:34:51.586193 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:34:51.587613 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:34:51.587919 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:34:51.587922 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:34:51.587923 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:34:51.588026 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:34:51.588198 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:34:51.588199 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:34:51.588199 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:34:51.588413 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:34:51.588558 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:51.588624 (+ 66us) server_negotiation.cc:299] Negotiation successful
0806 06:34:51.588749 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":80,"threads_started":1}
I20260806 06:34:51.596112 32558 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:34:51.596752 32558 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:34:51.603066 32558 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e83f6d77083d462cab65c301161189d1: Generated new TSK 0
I20260806 06:34:51.603230 32558 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.43.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab.
Entry for principal kudu/127.7.43.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab.
I20260806 06:34:51.629839 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.65:0
--local_ip_for_outbound_sockets=127.7.43.65
--webserver_interface=127.7.43.65
--webserver_port=0
--tserver_master_addrs=127.7.43.126:40111
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab
--principal=kudu/127.7.43.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:36017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:34:51.709233 32573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:34:51.709424 32573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:34:51.709455 32573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:34:51.710584 32573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:34:51.710623 32573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:34:51.710657 32573 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.65
I20260806 06:34:51.712088 32573 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:36017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab
--principal=kudu/127.7.43.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.43.65
--webserver_port=0
--tserver_master_addrs=127.7.43.126:40111
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32573
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.65
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:34:51.712313 32573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:34:51.712507 32573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:34:51.717109 32581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:34:51.717166 32573 server_base.cc:1034] running on GCE node
W20260806 06:34:51.717090 32584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:34:51.717092 32580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:34:51.717454 32573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:34:51.717684 32573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:34:51.718814 32573 hybrid_clock.cc:648] HybridClock initialized: now 1785998091718804 us; error 29 us; skew 500 ppm
Aug 06 06:34:51 dist-test-slave-6zbq krb5kdc[32418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998091, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:34:51.721179 32573 init.cc:377] Logged in from keytab as kudu/127.7.43.65@KRBTEST.COM (short username kudu)
I20260806 06:34:51.721550 32573 webserver.cc:460] Webserver started at http://127.7.43.65:43137/ using document root <none> and password file <none>
I20260806 06:34:51.721737 32573 fs_manager.cc:359] Metadata directory not provided
I20260806 06:34:51.721779 32573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:34:51.721889 32573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:34:51.722868 32573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/ts-0/data/instance:
uuid: "b6f8a30db45947c790d3b13680095439"
format_stamp: "Formatted at 2026-08-06 06:34:51 on dist-test-slave-6zbq"
server_key: "5fc6b4e035c9f0bca1a890c5d70a625b"
server_key_iv: "6a9689394bcf4a27d60ea943fc2e56d0"
server_key_version: "encryptionkey@0"
I20260806 06:34:51.723178 32573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/ts-0/wal/instance:
uuid: "b6f8a30db45947c790d3b13680095439"
format_stamp: "Formatted at 2026-08-06 06:34:51 on dist-test-slave-6zbq"
server_key: "5fc6b4e035c9f0bca1a890c5d70a625b"
server_key_iv: "6a9689394bcf4a27d60ea943fc2e56d0"
server_key_version: "encryptionkey@0"
I20260806 06:34:51.724429 32573 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:34:51.725044 32594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:51.725201 32573 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:34:51.725291 32573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/ts-0/wal
uuid: "b6f8a30db45947c790d3b13680095439"
format_stamp: "Formatted at 2026-08-06 06:34:51 on dist-test-slave-6zbq"
server_key: "5fc6b4e035c9f0bca1a890c5d70a625b"
server_key_iv: "6a9689394bcf4a27d60ea943fc2e56d0"
server_key_version: "encryptionkey@0"
I20260806 06:34:51.725343 32573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:34:51.745019 32573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:34:51.746709 32573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:34:51.746809 32573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:34:51.746960 32573 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:34:51.747225 32573 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:34:51.747267 32573 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:51.747285 32573 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:34:51.747306 32573 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:51.752604 32573 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.65:43505
I20260806 06:34:51.752785 32758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.65:43505 every 8 connection(s)
I20260806 06:34:51.752981 32573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
I20260806 06:34:51.754062 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 32573
I20260806 06:34:51.754163 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/ts-0/wal/instance
I20260806 06:34:51.754415 7341 external_mini_cluster.cc:1428] Setting key 75ec9eca1fe3da968b82baeffd204871
Aug 06 06:34:51 dist-test-slave-6zbq krb5kdc[32418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998091, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.65@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:34:51.761927 32541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:51.753625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:39167 (local address 127.7.43.126:40111)
0806 06:34:51.753670 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:51.753673 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:51.753865 (+ 192us) server_negotiation.cc:407] Connection header received
0806 06:34:51.754386 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:51.754388 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:51.754404 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:51.754474 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:34:51.755350 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:51.756003 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:51.756908 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:51.757125 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:51.757183 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:51.759384 (+ 2201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:34:51.759385 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:34:51.759385 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:34:51.759394 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:34:51.760619 (+ 1225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:34:51.761051 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:34:51.761052 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:34:51.761052 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:34:51.761107 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:34:51.761349 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:34:51.761349 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:34:51.761350 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:34:51.761443 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:34:51.761489 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:51.761796 (+ 307us) server_negotiation.cc:299] Negotiation successful
0806 06:34:51.761863 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:34:51.762138 32761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:51.753599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40111 (local address 127.7.43.65:39167)
0806 06:34:51.753760 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:51.753791 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:51.754278 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:51.754544 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:51.754548 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:51.755081 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:34:51.755264 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:51.755268 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:51.756077 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:51.756077 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:51.756825 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:51.756827 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:51.757196 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:51.757197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:51.757331 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:51.757745 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:34:51.757754 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:34:51.759291 (+ 1537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:34:51.760697 (+ 1406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:34:51.760703 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:34:51.760705 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:34:51.760985 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:34:51.761159 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:34:51.761160 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:34:51.761160 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:34:51.761294 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:34:51.761497 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:34:51.761502 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:34:51.761663 (+ 161us) client_negotiation.cc:769] Sending connection context
0806 06:34:51.761775 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 06:34:51.761928 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":102,"threads_started":1}
I20260806 06:34:51.762610 32759 heartbeater.cc:347] Connected to a master server at 127.7.43.126:40111
I20260806 06:34:51.762693 32759 heartbeater.cc:464] Registering TS with master...
I20260806 06:34:51.762854 32759 heartbeater.cc:511] Master 127.7.43.126:40111 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.7.43.66@KRBTEST.COM; defaulting to no policy
I20260806 06:34:51.763330 32479 ts_manager.cc:194] Registered new tserver with Master: b6f8a30db45947c790d3b13680095439 (127.7.43.65:43505)
I20260806 06:34:51.763945 32479 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.65@KRBTEST.COM'} at 127.7.43.65:39167
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab.
Entry for principal kudu/127.7.43.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab.
I20260806 06:34:51.778146 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.66:0
--local_ip_for_outbound_sockets=127.7.43.66
--webserver_interface=127.7.43.66
--webserver_port=0
--tserver_master_addrs=127.7.43.126:40111
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab
--principal=kudu/127.7.43.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:36017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:34:51.855855 32764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:34:51.856027 32764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:34:51.856048 32764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:34:51.857136 32764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:34:51.857174 32764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:34:51.857198 32764 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.66
I20260806 06:34:51.858564 32764 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:36017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab
--principal=kudu/127.7.43.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.43.66
--webserver_port=0
--tserver_master_addrs=127.7.43.126:40111
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32764
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.66
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:34:51.858734 32764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:34:51.858894 32764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:34:51.863373 302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:34:51.863395 306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:34:51.863410 303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:34:51.863519 32764 server_base.cc:1034] running on GCE node
I20260806 06:34:51.863824 32764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:34:51.864006 32764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:34:51.865149 32764 hybrid_clock.cc:648] HybridClock initialized: now 1785998091865138 us; error 28 us; skew 500 ppm
Aug 06 06:34:51 dist-test-slave-6zbq krb5kdc[32418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998091, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:34:51.867429 32764 init.cc:377] Logged in from keytab as kudu/127.7.43.66@KRBTEST.COM (short username kudu)
I20260806 06:34:51.867798 32764 webserver.cc:460] Webserver started at http://127.7.43.66:44031/ using document root <none> and password file <none>
I20260806 06:34:51.867976 32764 fs_manager.cc:359] Metadata directory not provided
I20260806 06:34:51.868022 32764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:34:51.868139 32764 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:34:51.869067 32764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/ts-1/data/instance:
uuid: "65c21a86fd864c9da1cc0b52228a5132"
format_stamp: "Formatted at 2026-08-06 06:34:51 on dist-test-slave-6zbq"
server_key: "f2781bcf9cf584aa1db42f701ef405d7"
server_key_iv: "e5754081942d0d2b34e7f019bc1bc632"
server_key_version: "encryptionkey@0"
I20260806 06:34:51.869390 32764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/ts-1/wal/instance:
uuid: "65c21a86fd864c9da1cc0b52228a5132"
format_stamp: "Formatted at 2026-08-06 06:34:51 on dist-test-slave-6zbq"
server_key: "f2781bcf9cf584aa1db42f701ef405d7"
server_key_iv: "e5754081942d0d2b34e7f019bc1bc632"
server_key_version: "encryptionkey@0"
I20260806 06:34:51.870594 32764 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:34:51.871192 319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:51.871341 32764 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:34:51.871420 32764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/ts-1/wal
uuid: "65c21a86fd864c9da1cc0b52228a5132"
format_stamp: "Formatted at 2026-08-06 06:34:51 on dist-test-slave-6zbq"
server_key: "f2781bcf9cf584aa1db42f701ef405d7"
server_key_iv: "e5754081942d0d2b34e7f019bc1bc632"
server_key_version: "encryptionkey@0"
I20260806 06:34:51.871476 32764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:34:51.885183 32764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:34:51.887146 32764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:34:51.887280 32764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:34:51.887466 32764 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:34:51.887718 32764 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:34:51.887748 32764 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:51.887775 32764 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:34:51.887791 32764 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:51.893132 32764 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.66:42149
I20260806 06:34:51.893198 490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.66:42149 every 8 connection(s)
I20260806 06:34:51.893455 32764 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
Aug 06 06:34:51 dist-test-slave-6zbq krb5kdc[32418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998091, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.66@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:34:51.901166 32541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:51.894049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:36615 (local address 127.7.43.126:40111)
0806 06:34:51.894100 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:51.894102 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:51.894277 (+ 175us) server_negotiation.cc:407] Connection header received
0806 06:34:51.894762 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:51.894763 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:51.894773 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:51.894823 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:34:51.895629 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:51.896111 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:51.897021 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:51.897196 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:51.897239 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:51.899042 (+ 1803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:34:51.899043 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:34:51.899043 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:34:51.899050 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:34:51.900213 (+ 1163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:34:51.900505 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:34:51.900505 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:34:51.900506 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:34:51.900551 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:34:51.900718 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:34:51.900719 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:34:51.900719 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:34:51.900816 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:34:51.900859 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:51.901038 (+ 179us) server_negotiation.cc:299] Negotiation successful
0806 06:34:51.901105 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 06:34:51.901304 496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:51.894025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40111 (local address 127.7.43.66:36615)
0806 06:34:51.894167 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:51.894204 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:51.894672 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:51.894889 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:51.894892 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:51.895397 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:34:51.895563 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:51.895567 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:51.896180 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:51.896181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:51.896959 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:51.896960 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:51.897252 (+ 292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:51.897253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:51.897346 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:51.897688 (+ 342us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:34:51.897696 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:34:51.898953 (+ 1257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:34:51.900276 (+ 1323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:34:51.900279 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:34:51.900280 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:34:51.900457 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:34:51.900592 (+ 135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:34:51.900593 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:34:51.900593 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:34:51.900679 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:34:51.900856 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:34:51.900858 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:34:51.900951 (+ 93us) client_negotiation.cc:769] Sending connection context
0806 06:34:51.901022 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 06:34:51.901140 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":64,"threads_started":1}
I20260806 06:34:51.901700 493 heartbeater.cc:347] Connected to a master server at 127.7.43.126:40111
I20260806 06:34:51.901777 493 heartbeater.cc:464] Registering TS with master...
I20260806 06:34:51.901930 493 heartbeater.cc:511] Master 127.7.43.126:40111 requested a full tablet report, sending...
I20260806 06:34:51.902160 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 32764
I20260806 06:34:51.902280 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/ts-1/wal/instance
I20260806 06:34:51.902285 32479 ts_manager.cc:194] Registered new tserver with Master: 65c21a86fd864c9da1cc0b52228a5132 (127.7.43.66:42149)
I20260806 06:34:51.902513 7341 external_mini_cluster.cc:1428] Setting key d85231e5b6dfae80379e055a34de2ffd
I20260806 06:34:51.902761 32479 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.66@KRBTEST.COM'} at 127.7.43.66:36615
WARNING: no policy specified for kudu/127.7.43.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab.
Entry for principal kudu/127.7.43.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab.
I20260806 06:34:51.924808 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.67:0
--local_ip_for_outbound_sockets=127.7.43.67
--webserver_interface=127.7.43.67
--webserver_port=0
--tserver_master_addrs=127.7.43.126:40111
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab
--principal=kudu/127.7.43.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:36017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:34:52.002242 503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:34:52.002435 503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:34:52.002465 503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:34:52.003579 503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:34:52.003618 503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:34:52.003661 503 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.67
I20260806 06:34:52.005038 503 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:36017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab
--principal=kudu/127.7.43.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.43.67
--webserver_port=0
--tserver_master_addrs=127.7.43.126:40111
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.503
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.67
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:34:52.005242 503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:34:52.005437 503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:34:52.009965 510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:34:52.010000 514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:34:52.010051 511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:34:52.010053 503 server_base.cc:1034] running on GCE node
I20260806 06:34:52.010357 503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:34:52.010578 503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:34:52.011721 503 hybrid_clock.cc:648] HybridClock initialized: now 1785998092011700 us; error 32 us; skew 500 ppm
Aug 06 06:34:52 dist-test-slave-6zbq krb5kdc[32418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998092, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:34:52.013940 503 init.cc:377] Logged in from keytab as kudu/127.7.43.67@KRBTEST.COM (short username kudu)
I20260806 06:34:52.014394 503 webserver.cc:460] Webserver started at http://127.7.43.67:41523/ using document root <none> and password file <none>
I20260806 06:34:52.014596 503 fs_manager.cc:359] Metadata directory not provided
I20260806 06:34:52.014642 503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:34:52.014760 503 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:34:52.015688 503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/ts-2/data/instance:
uuid: "2d547e2d6d8946bea17fcdc9735ad8d3"
format_stamp: "Formatted at 2026-08-06 06:34:52 on dist-test-slave-6zbq"
server_key: "cee4a79761b65e3dcade541239cc0e69"
server_key_iv: "ca3b581758786d895d2f4ceb78385953"
server_key_version: "encryptionkey@0"
I20260806 06:34:52.016005 503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/ts-2/wal/instance:
uuid: "2d547e2d6d8946bea17fcdc9735ad8d3"
format_stamp: "Formatted at 2026-08-06 06:34:52 on dist-test-slave-6zbq"
server_key: "cee4a79761b65e3dcade541239cc0e69"
server_key_iv: "ca3b581758786d895d2f4ceb78385953"
server_key_version: "encryptionkey@0"
I20260806 06:34:52.017163 503 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:34:52.017762 528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:52.017910 503 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:52.017987 503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/ts-2/wal
uuid: "2d547e2d6d8946bea17fcdc9735ad8d3"
format_stamp: "Formatted at 2026-08-06 06:34:52 on dist-test-slave-6zbq"
server_key: "cee4a79761b65e3dcade541239cc0e69"
server_key_iv: "ca3b581758786d895d2f4ceb78385953"
server_key_version: "encryptionkey@0"
I20260806 06:34:52.018051 503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:34:52.029640 503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:34:52.031338 503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:34:52.031461 503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:34:52.031646 503 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:34:52.031904 503 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:34:52.031936 503 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:52.031962 503 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:34:52.031980 503 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:52.037657 503 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.67:39975
I20260806 06:34:52.037709 689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.67:39975 every 8 connection(s)
I20260806 06:34:52.037961 503 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
I20260806 06:34:52.038640 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 503
I20260806 06:34:52.038758 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/ts-2/wal/instance
I20260806 06:34:52.038972 7341 external_mini_cluster.cc:1428] Setting key e4ce8dbd4b9c7417e0f47e3813e62443
Aug 06 06:34:52 dist-test-slave-6zbq krb5kdc[32418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998092, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.67@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:34:52.045881 32541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:52.038616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:52323 (local address 127.7.43.126:40111)
0806 06:34:52.038678 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:52.038681 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:52.038906 (+ 225us) server_negotiation.cc:407] Connection header received
0806 06:34:52.039404 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:52.039405 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:52.039416 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:52.039471 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:34:52.040371 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.040848 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.041735 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.041908 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.041951 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:52.043817 (+ 1866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:34:52.043818 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:34:52.043818 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:34:52.043824 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:34:52.044882 (+ 1058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:34:52.045215 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:34:52.045216 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:34:52.045216 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:34:52.045265 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:34:52.045435 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:34:52.045435 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:34:52.045435 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:34:52.045525 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:34:52.045567 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:52.045760 (+ 193us) server_negotiation.cc:299] Negotiation successful
0806 06:34:52.045820 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 06:34:52.046029 693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:52.038602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40111 (local address 127.7.43.67:52323)
0806 06:34:52.038778 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:52.038817 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:52.039319 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:52.039543 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:52.039547 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:52.040125 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:34:52.040295 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:52.040300 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.040909 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.040910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:52.041672 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:52.041674 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.041960 (+ 286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.041960 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:52.042046 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:52.042505 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:34:52.042513 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:34:52.043739 (+ 1226us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:34:52.044944 (+ 1205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:34:52.044950 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:34:52.044951 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:34:52.045154 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:34:52.045310 (+ 156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:34:52.045311 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:34:52.045311 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:34:52.045394 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:34:52.045564 (+ 170us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:34:52.045567 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:34:52.045670 (+ 103us) client_negotiation.cc:769] Sending connection context
0806 06:34:52.045742 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 06:34:52.045849 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":82,"threads_started":1}
I20260806 06:34:52.046427 690 heartbeater.cc:347] Connected to a master server at 127.7.43.126:40111
I20260806 06:34:52.046509 690 heartbeater.cc:464] Registering TS with master...
I20260806 06:34:52.046676 690 heartbeater.cc:511] Master 127.7.43.126:40111 requested a full tablet report, sending...
I20260806 06:34:52.047014 32478 ts_manager.cc:194] Registered new tserver with Master: 2d547e2d6d8946bea17fcdc9735ad8d3 (127.7.43.67:39975)
I20260806 06:34:52.047360 32478 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.67@KRBTEST.COM'} at 127.7.43.67:52323
I20260806 06:34:52.050539 7341 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:34:52.056766 32541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:52.051731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44838 (local address 127.7.43.126:40111)
0806 06:34:52.051779 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:52.051781 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:52.051886 (+ 105us) server_negotiation.cc:407] Connection header received
0806 06:34:52.051946 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:52.051947 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:52.051955 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:52.051999 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:34:52.052507 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.052939 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.053628 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.053803 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.053853 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:52.054865 (+ 1012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:34:52.054866 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:34:52.054867 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:34:52.054873 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:34:52.055806 (+ 933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:34:52.056085 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:34:52.056086 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:34:52.056086 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:34:52.056147 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:34:52.056304 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:34:52.056305 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:34:52.056305 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:34:52.056428 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:34:52.056509 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:52.056617 (+ 108us) server_negotiation.cc:299] Negotiation successful
0806 06:34:52.056685 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 06:34:52.057940 32478 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44838:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:34:52.059414 32478 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:34:52.065047 714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:52.061846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37464 (local address 127.7.43.67:39975)
0806 06:34:52.061995 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:52.061999 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:52.062007 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:34:52.062037 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:52.062039 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:52.062053 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:52.062147 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:34:52.062387 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.063303 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.064170 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.064696 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.064756 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:52.064807 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:52.064943 (+ 136us) server_negotiation.cc:299] Negotiation successful
0806 06:34:52.064976 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":74,"threads_started":1}
I20260806 06:34:52.065042 711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:52.061459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.67:39975 (local address 127.0.0.1:37464)
0806 06:34:52.061568 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:52.061575 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:52.061607 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:52.062258 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:52.062258 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:52.062260 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:34:52.062296 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:52.062297 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.063384 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.063385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:52.064077 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:52.064079 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.064772 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.064773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:52.064852 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:52.064864 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 06:34:52.064924 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 06:34:52.064973 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":77,"threads_started":1}
I20260806 06:34:52.065222 707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:52.061256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.66:42149 (local address 127.0.0.1:37922)
0806 06:34:52.061369 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:52.061402 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:52.061463 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:52.061918 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:52.061919 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:52.061921 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:34:52.061950 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:52.061951 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.063114 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.063115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:52.064155 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:52.064157 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.065018 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.065019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:52.065079 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:52.065083 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:34:52.065113 (+ 30us) client_negotiation.cc:240] Negotiation successful
0806 06:34:52.065149 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":72,"threads_started":1}
I20260806 06:34:52.065549 712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:52.061619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37922 (local address 127.7.43.66:42149)
0806 06:34:52.061788 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:52.061792 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:52.061802 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:34:52.061838 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:52.061840 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:52.061854 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:52.061924 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:34:52.061991 (+ 67us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.063016 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.064237 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.064918 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.065341 (+ 423us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:52.065404 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:52.065429 (+ 25us) server_negotiation.cc:299] Negotiation successful
0806 06:34:52.065471 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":93,"threads_started":1}
I20260806 06:34:52.065639 708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:52.061257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:43505 (local address 127.0.0.1:60394)
0806 06:34:52.061406 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:52.061414 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:52.061463 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:52.061751 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:52.061753 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:52.061755 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:34:52.061814 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:52.061816 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.062870 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.062871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:52.064121 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:52.064124 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.065069 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.065070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:52.065481 (+ 411us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:52.065485 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:34:52.065542 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 06:34:52.065578 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":3,"thread_start_us":67,"threads_started":1}
I20260806 06:34:52.065698 710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:52.061385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60394 (local address 127.7.43.65:43505)
0806 06:34:52.061572 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:52.061576 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:52.061587 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:34:52.061639 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:52.061641 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:52.061656 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:52.061741 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:34:52.061886 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.062780 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.064221 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.064918 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.064986 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:52.065041 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:52.065570 (+ 529us) server_negotiation.cc:299] Negotiation successful
0806 06:34:52.065613 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":70,"threads_started":1}
I20260806 06:34:52.065806 597 tablet_service.cc:1467] Processing CreateTablet for tablet fcdfe050b0e444b5ad7777a8eda2e029 (DEFAULT_TABLE table=test-table [id=07b81d266b0649af81c092651c5dc1f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:34:52.066107 597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcdfe050b0e444b5ad7777a8eda2e029. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:34:52.066747 398 tablet_service.cc:1467] Processing CreateTablet for tablet fcdfe050b0e444b5ad7777a8eda2e029 (DEFAULT_TABLE table=test-table [id=07b81d266b0649af81c092651c5dc1f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:34:52.067044 398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcdfe050b0e444b5ad7777a8eda2e029. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:34:52.068640 717 tablet_bootstrap.cc:492] T fcdfe050b0e444b5ad7777a8eda2e029 P 2d547e2d6d8946bea17fcdc9735ad8d3: Bootstrap starting.
I20260806 06:34:52.069334 717 tablet_bootstrap.cc:654] T fcdfe050b0e444b5ad7777a8eda2e029 P 2d547e2d6d8946bea17fcdc9735ad8d3: Neither blocks nor log segments found. Creating new log.
I20260806 06:34:52.069669 719 tablet_bootstrap.cc:492] T fcdfe050b0e444b5ad7777a8eda2e029 P 65c21a86fd864c9da1cc0b52228a5132: Bootstrap starting.
I20260806 06:34:52.069906 717 log.cc:824] T fcdfe050b0e444b5ad7777a8eda2e029 P 2d547e2d6d8946bea17fcdc9735ad8d3: Log is configured to *not* fsync() on all Append() calls
I20260806 06:34:52.070065 32664 tablet_service.cc:1467] Processing CreateTablet for tablet fcdfe050b0e444b5ad7777a8eda2e029 (DEFAULT_TABLE table=test-table [id=07b81d266b0649af81c092651c5dc1f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:34:52.070315 719 tablet_bootstrap.cc:654] T fcdfe050b0e444b5ad7777a8eda2e029 P 65c21a86fd864c9da1cc0b52228a5132: Neither blocks nor log segments found. Creating new log.
I20260806 06:34:52.070422 32664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcdfe050b0e444b5ad7777a8eda2e029. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:34:52.070699 717 tablet_bootstrap.cc:492] T fcdfe050b0e444b5ad7777a8eda2e029 P 2d547e2d6d8946bea17fcdc9735ad8d3: No bootstrap required, opened a new log
I20260806 06:34:52.070820 717 ts_tablet_manager.cc:1397] T fcdfe050b0e444b5ad7777a8eda2e029 P 2d547e2d6d8946bea17fcdc9735ad8d3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:34:52.070817 719 log.cc:824] T fcdfe050b0e444b5ad7777a8eda2e029 P 65c21a86fd864c9da1cc0b52228a5132: Log is configured to *not* fsync() on all Append() calls
I20260806 06:34:52.071413 719 tablet_bootstrap.cc:492] T fcdfe050b0e444b5ad7777a8eda2e029 P 65c21a86fd864c9da1cc0b52228a5132: No bootstrap required, opened a new log
I20260806 06:34:52.071468 719 ts_tablet_manager.cc:1397] T fcdfe050b0e444b5ad7777a8eda2e029 P 65c21a86fd864c9da1cc0b52228a5132: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:34:52.072669 717 raft_consensus.cc:348] T fcdfe050b0e444b5ad7777a8eda2e029 P 2d547e2d6d8946bea17fcdc9735ad8d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f8a30db45947c790d3b13680095439" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43505 } } peers { permanent_uuid: "65c21a86fd864c9da1cc0b52228a5132" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 42149 } } peers { permanent_uuid: "2d547e2d6d8946bea17fcdc9735ad8d3" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 39975 } }
I20260806 06:34:52.072924 717 raft_consensus.cc:374] T fcdfe050b0e444b5ad7777a8eda2e029 P 2d547e2d6d8946bea17fcdc9735ad8d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:34:52.072829 719 raft_consensus.cc:348] T fcdfe050b0e444b5ad7777a8eda2e029 P 65c21a86fd864c9da1cc0b52228a5132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f8a30db45947c790d3b13680095439" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43505 } } peers { permanent_uuid: "65c21a86fd864c9da1cc0b52228a5132" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 42149 } } peers { permanent_uuid: "2d547e2d6d8946bea17fcdc9735ad8d3" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 39975 } }
I20260806 06:34:52.072999 717 raft_consensus.cc:729] T fcdfe050b0e444b5ad7777a8eda2e029 P 2d547e2d6d8946bea17fcdc9735ad8d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d547e2d6d8946bea17fcdc9735ad8d3, State: Initialized, Role: FOLLOWER
I20260806 06:34:52.073000 719 raft_consensus.cc:374] T fcdfe050b0e444b5ad7777a8eda2e029 P 65c21a86fd864c9da1cc0b52228a5132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:34:52.073041 719 raft_consensus.cc:729] T fcdfe050b0e444b5ad7777a8eda2e029 P 65c21a86fd864c9da1cc0b52228a5132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65c21a86fd864c9da1cc0b52228a5132, State: Initialized, Role: FOLLOWER
I20260806 06:34:52.073149 717 consensus_queue.cc:260] T fcdfe050b0e444b5ad7777a8eda2e029 P 2d547e2d6d8946bea17fcdc9735ad8d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f8a30db45947c790d3b13680095439" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43505 } } peers { permanent_uuid: "65c21a86fd864c9da1cc0b52228a5132" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 42149 } } peers { permanent_uuid: "2d547e2d6d8946bea17fcdc9735ad8d3" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 39975 } }
I20260806 06:34:52.073170 719 consensus_queue.cc:260] T fcdfe050b0e444b5ad7777a8eda2e029 P 65c21a86fd864c9da1cc0b52228a5132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f8a30db45947c790d3b13680095439" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43505 } } peers { permanent_uuid: "65c21a86fd864c9da1cc0b52228a5132" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 42149 } } peers { permanent_uuid: "2d547e2d6d8946bea17fcdc9735ad8d3" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 39975 } }
I20260806 06:34:52.073419 719 ts_tablet_manager.cc:1428] T fcdfe050b0e444b5ad7777a8eda2e029 P 65c21a86fd864c9da1cc0b52228a5132: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:34:52.073475 717 ts_tablet_manager.cc:1428] T fcdfe050b0e444b5ad7777a8eda2e029 P 2d547e2d6d8946bea17fcdc9735ad8d3: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 06:34:52.073495 726 tablet_bootstrap.cc:492] T fcdfe050b0e444b5ad7777a8eda2e029 P b6f8a30db45947c790d3b13680095439: Bootstrap starting.
I20260806 06:34:52.073596 493 heartbeater.cc:503] Master 127.7.43.126:40111 was elected leader, sending a full tablet report...
I20260806 06:34:52.073673 690 heartbeater.cc:503] Master 127.7.43.126:40111 was elected leader, sending a full tablet report...
I20260806 06:34:52.074087 726 tablet_bootstrap.cc:654] T fcdfe050b0e444b5ad7777a8eda2e029 P b6f8a30db45947c790d3b13680095439: Neither blocks nor log segments found. Creating new log.
I20260806 06:34:52.074671 726 log.cc:824] T fcdfe050b0e444b5ad7777a8eda2e029 P b6f8a30db45947c790d3b13680095439: Log is configured to *not* fsync() on all Append() calls
I20260806 06:34:52.075317 726 tablet_bootstrap.cc:492] T fcdfe050b0e444b5ad7777a8eda2e029 P b6f8a30db45947c790d3b13680095439: No bootstrap required, opened a new log
I20260806 06:34:52.075373 726 ts_tablet_manager.cc:1397] T fcdfe050b0e444b5ad7777a8eda2e029 P b6f8a30db45947c790d3b13680095439: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:34:52.076618 726 raft_consensus.cc:348] T fcdfe050b0e444b5ad7777a8eda2e029 P b6f8a30db45947c790d3b13680095439 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f8a30db45947c790d3b13680095439" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43505 } } peers { permanent_uuid: "65c21a86fd864c9da1cc0b52228a5132" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 42149 } } peers { permanent_uuid: "2d547e2d6d8946bea17fcdc9735ad8d3" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 39975 } }
I20260806 06:34:52.076749 726 raft_consensus.cc:374] T fcdfe050b0e444b5ad7777a8eda2e029 P b6f8a30db45947c790d3b13680095439 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:34:52.076771 726 raft_consensus.cc:729] T fcdfe050b0e444b5ad7777a8eda2e029 P b6f8a30db45947c790d3b13680095439 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6f8a30db45947c790d3b13680095439, State: Initialized, Role: FOLLOWER
I20260806 06:34:52.076874 726 consensus_queue.cc:260] T fcdfe050b0e444b5ad7777a8eda2e029 P b6f8a30db45947c790d3b13680095439 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f8a30db45947c790d3b13680095439" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43505 } } peers { permanent_uuid: "65c21a86fd864c9da1cc0b52228a5132" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 42149 } } peers { permanent_uuid: "2d547e2d6d8946bea17fcdc9735ad8d3" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 39975 } }
I20260806 06:34:52.077116 726 ts_tablet_manager.cc:1428] T fcdfe050b0e444b5ad7777a8eda2e029 P b6f8a30db45947c790d3b13680095439: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:34:52.077137 32759 heartbeater.cc:503] Master 127.7.43.126:40111 was elected leader, sending a full tablet report...
W20260806 06:34:52.143973 495 tablet.cc:2369] T fcdfe050b0e444b5ad7777a8eda2e029 P 65c21a86fd864c9da1cc0b52228a5132: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:34:52.184566 727 raft_consensus.cc:482] T fcdfe050b0e444b5ad7777a8eda2e029 P 2d547e2d6d8946bea17fcdc9735ad8d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:34:52.184674 727 raft_consensus.cc:504] T fcdfe050b0e444b5ad7777a8eda2e029 P 2d547e2d6d8946bea17fcdc9735ad8d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f8a30db45947c790d3b13680095439" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43505 } } peers { permanent_uuid: "65c21a86fd864c9da1cc0b52228a5132" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 42149 } } peers { permanent_uuid: "2d547e2d6d8946bea17fcdc9735ad8d3" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 39975 } }
I20260806 06:34:52.184976 727 leader_election.cc:288] T fcdfe050b0e444b5ad7777a8eda2e029 P 2d547e2d6d8946bea17fcdc9735ad8d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b6f8a30db45947c790d3b13680095439 (127.7.43.65:43505), 65c21a86fd864c9da1cc0b52228a5132 (127.7.43.66:42149)
I20260806 06:34:52.187538 693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:52.185129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:43505 (local address 127.7.43.67:49725)
0806 06:34:52.185177 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:52.185190 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:52.185234 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:52.185368 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:52.185369 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:52.185371 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:34:52.185435 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:52.185437 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.186036 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.186037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:52.186792 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:52.186794 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.187312 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.187313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:52.187381 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:52.187384 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:34:52.187429 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 06:34:52.187466 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 06:34:52.187538 710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:52.185220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:49725 (local address 127.7.43.65:43505)
0806 06:34:52.185263 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:52.185264 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:52.185273 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:34:52.185297 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:52.185298 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:52.185309 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:52.185358 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:34:52.185492 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.185964 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.186872 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.187240 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.187303 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:52.187315 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:52.187443 (+ 128us) server_negotiation.cc:299] Negotiation successful
0806 06:34:52.187479 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 06:34:52.187769 712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:52.185237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:55069 (local address 127.7.43.66:42149)
0806 06:34:52.185282 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:52.185284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:52.185403 (+ 119us) server_negotiation.cc:407] Connection header received
0806 06:34:52.185471 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:52.185472 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:52.185484 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:52.185525 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:34:52.185662 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.186126 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.187040 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.187418 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.187479 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:52.187489 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:52.187646 (+ 157us) server_negotiation.cc:299] Negotiation successful
0806 06:34:52.187706 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:34:52.187911 32699 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcdfe050b0e444b5ad7777a8eda2e029" candidate_uuid: "2d547e2d6d8946bea17fcdc9735ad8d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6f8a30db45947c790d3b13680095439" is_pre_election: true
I20260806 06:34:52.188000 732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:52.185139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.66:42149 (local address 127.7.43.67:55069)
0806 06:34:52.185336 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:52.185344 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:52.185409 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:52.185537 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:52.185538 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:52.185539 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:34:52.185604 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:52.185606 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.186203 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.186204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:52.186955 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:52.186957 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.187495 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.187496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:52.187560 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:52.187564 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:34:52.187881 (+ 317us) client_negotiation.cc:240] Negotiation successful
0806 06:34:52.187922 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":20,"thread_start_us":102,"threads_started":1}
I20260806 06:34:52.188052 32699 raft_consensus.cc:2454] T fcdfe050b0e444b5ad7777a8eda2e029 P b6f8a30db45947c790d3b13680095439 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d547e2d6d8946bea17fcdc9735ad8d3 in term 0.
I20260806 06:34:52.188283 533 leader_election.cc:302] T fcdfe050b0e444b5ad7777a8eda2e029 P 2d547e2d6d8946bea17fcdc9735ad8d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d547e2d6d8946bea17fcdc9735ad8d3, b6f8a30db45947c790d3b13680095439; no voters:
I20260806 06:34:52.188321 431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcdfe050b0e444b5ad7777a8eda2e029" candidate_uuid: "2d547e2d6d8946bea17fcdc9735ad8d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65c21a86fd864c9da1cc0b52228a5132" is_pre_election: true
I20260806 06:34:52.188414 727 raft_consensus.cc:2781] T fcdfe050b0e444b5ad7777a8eda2e029 P 2d547e2d6d8946bea17fcdc9735ad8d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:34:52.188458 431 raft_consensus.cc:2454] T fcdfe050b0e444b5ad7777a8eda2e029 P 65c21a86fd864c9da1cc0b52228a5132 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d547e2d6d8946bea17fcdc9735ad8d3 in term 0.
I20260806 06:34:52.188484 727 raft_consensus.cc:482] T fcdfe050b0e444b5ad7777a8eda2e029 P 2d547e2d6d8946bea17fcdc9735ad8d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:34:52.188508 727 raft_consensus.cc:3037] T fcdfe050b0e444b5ad7777a8eda2e029 P 2d547e2d6d8946bea17fcdc9735ad8d3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:34:52.189224 727 raft_consensus.cc:504] T fcdfe050b0e444b5ad7777a8eda2e029 P 2d547e2d6d8946bea17fcdc9735ad8d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f8a30db45947c790d3b13680095439" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43505 } } peers { permanent_uuid: "65c21a86fd864c9da1cc0b52228a5132" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 42149 } } peers { permanent_uuid: "2d547e2d6d8946bea17fcdc9735ad8d3" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 39975 } }
I20260806 06:34:52.189333 727 leader_election.cc:288] T fcdfe050b0e444b5ad7777a8eda2e029 P 2d547e2d6d8946bea17fcdc9735ad8d3 [CANDIDATE]: Term 1 election: Requested vote from peers b6f8a30db45947c790d3b13680095439 (127.7.43.65:43505), 65c21a86fd864c9da1cc0b52228a5132 (127.7.43.66:42149)
I20260806 06:34:52.189465 32699 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcdfe050b0e444b5ad7777a8eda2e029" candidate_uuid: "2d547e2d6d8946bea17fcdc9735ad8d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6f8a30db45947c790d3b13680095439"
I20260806 06:34:52.189525 431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcdfe050b0e444b5ad7777a8eda2e029" candidate_uuid: "2d547e2d6d8946bea17fcdc9735ad8d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65c21a86fd864c9da1cc0b52228a5132"
I20260806 06:34:52.189549 32699 raft_consensus.cc:3037] T fcdfe050b0e444b5ad7777a8eda2e029 P b6f8a30db45947c790d3b13680095439 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:34:52.189600 431 raft_consensus.cc:3037] T fcdfe050b0e444b5ad7777a8eda2e029 P 65c21a86fd864c9da1cc0b52228a5132 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:34:52.190439 431 raft_consensus.cc:2454] T fcdfe050b0e444b5ad7777a8eda2e029 P 65c21a86fd864c9da1cc0b52228a5132 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d547e2d6d8946bea17fcdc9735ad8d3 in term 1.
I20260806 06:34:52.190439 32699 raft_consensus.cc:2454] T fcdfe050b0e444b5ad7777a8eda2e029 P b6f8a30db45947c790d3b13680095439 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d547e2d6d8946bea17fcdc9735ad8d3 in term 1.
I20260806 06:34:52.190589 533 leader_election.cc:302] T fcdfe050b0e444b5ad7777a8eda2e029 P 2d547e2d6d8946bea17fcdc9735ad8d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d547e2d6d8946bea17fcdc9735ad8d3, b6f8a30db45947c790d3b13680095439; no voters:
I20260806 06:34:52.190701 727 raft_consensus.cc:2781] T fcdfe050b0e444b5ad7777a8eda2e029 P 2d547e2d6d8946bea17fcdc9735ad8d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:34:52.190831 727 raft_consensus.cc:686] T fcdfe050b0e444b5ad7777a8eda2e029 P 2d547e2d6d8946bea17fcdc9735ad8d3 [term 1 LEADER]: Becoming Leader. State: Replica: 2d547e2d6d8946bea17fcdc9735ad8d3, State: Running, Role: LEADER
I20260806 06:34:52.190910 727 consensus_queue.cc:237] T fcdfe050b0e444b5ad7777a8eda2e029 P 2d547e2d6d8946bea17fcdc9735ad8d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f8a30db45947c790d3b13680095439" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43505 } } peers { permanent_uuid: "65c21a86fd864c9da1cc0b52228a5132" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 42149 } } peers { permanent_uuid: "2d547e2d6d8946bea17fcdc9735ad8d3" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 39975 } }
I20260806 06:34:52.191669 32478 catalog_manager.cc:5495] T fcdfe050b0e444b5ad7777a8eda2e029 P 2d547e2d6d8946bea17fcdc9735ad8d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d547e2d6d8946bea17fcdc9735ad8d3 (127.7.43.67). New cstate: current_term: 1 leader_uuid: "2d547e2d6d8946bea17fcdc9735ad8d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f8a30db45947c790d3b13680095439" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "65c21a86fd864c9da1cc0b52228a5132" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 42149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d547e2d6d8946bea17fcdc9735ad8d3" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 39975 } health_report { overall_health: HEALTHY } } }
I20260806 06:34:52.207000 714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:52.205106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37480 (local address 127.7.43.67:39975)
0806 06:34:52.205154 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:52.205156 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:52.205164 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:34:52.205193 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:52.205193 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:52.205208 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:52.205270 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:34:52.205353 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.205765 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.206459 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.206628 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.206680 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:52.206779 (+ 99us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:34:52.206872 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:34:52.206930 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:52.206943 (+ 13us) server_negotiation.cc:299] Negotiation successful
0806 06:34:52.206956 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 06:34:52.208532 32699 raft_consensus.cc:1261] T fcdfe050b0e444b5ad7777a8eda2e029 P b6f8a30db45947c790d3b13680095439 [term 1 FOLLOWER]: Refusing update from remote peer 2d547e2d6d8946bea17fcdc9735ad8d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:34:52.208532 431 raft_consensus.cc:1261] T fcdfe050b0e444b5ad7777a8eda2e029 P 65c21a86fd864c9da1cc0b52228a5132 [term 1 FOLLOWER]: Refusing update from remote peer 2d547e2d6d8946bea17fcdc9735ad8d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:34:52.208832 727 consensus_queue.cc:1035] T fcdfe050b0e444b5ad7777a8eda2e029 P 2d547e2d6d8946bea17fcdc9735ad8d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6f8a30db45947c790d3b13680095439" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 43505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:34:52.208971 733 consensus_queue.cc:1035] T fcdfe050b0e444b5ad7777a8eda2e029 P 2d547e2d6d8946bea17fcdc9735ad8d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "65c21a86fd864c9da1cc0b52228a5132" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 42149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:34:52.211472 740 mvcc.cc:205] Tried to move back new op lower bound from 7315448185683881984 to 7315448185614573568. Current Snapshot: MvccSnapshot[applied={T|T < 7315448185683881984}]
I20260806 06:34:52.212155 739 mvcc.cc:205] Tried to move back new op lower bound from 7315448185683881984 to 7315448185614573568. Current Snapshot: MvccSnapshot[applied={T|T < 7315448185683881984}]
I20260806 06:34:52.213121 32478 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44838:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:34:52.213204 32478 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44838:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:34:52.213851 32478 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e83f6d77083d462cab65c301161189d1: Sending DeleteTablet for 3 replicas of tablet fcdfe050b0e444b5ad7777a8eda2e029
I20260806 06:34:52.214166 32664 tablet_service.cc:1514] Processing DeleteTablet for tablet fcdfe050b0e444b5ad7777a8eda2e029 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:34:52 UTC) from {username='kudu', principal='kudu/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:60394
I20260806 06:34:52.214170 398 tablet_service.cc:1514] Processing DeleteTablet for tablet fcdfe050b0e444b5ad7777a8eda2e029 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:34:52 UTC) from {username='kudu', principal='kudu/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:37922
I20260806 06:34:52.214205 597 tablet_service.cc:1514] Processing DeleteTablet for tablet fcdfe050b0e444b5ad7777a8eda2e029 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:34:52 UTC) from {username='kudu', principal='kudu/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:37464
I20260806 06:34:52.214485 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 32573
I20260806 06:34:52.214491 753 tablet_replica.cc:330] T fcdfe050b0e444b5ad7777a8eda2e029 P 65c21a86fd864c9da1cc0b52228a5132: stopping tablet replica
I20260806 06:34:52.221192 753 raft_consensus.cc:2229] T fcdfe050b0e444b5ad7777a8eda2e029 P 65c21a86fd864c9da1cc0b52228a5132 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:34:52.221323 753 raft_consensus.cc:2258] T fcdfe050b0e444b5ad7777a8eda2e029 P 65c21a86fd864c9da1cc0b52228a5132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:34:52.221755 753 ts_tablet_manager.cc:1905] T fcdfe050b0e444b5ad7777a8eda2e029 P 65c21a86fd864c9da1cc0b52228a5132: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:34:52.223270 753 ts_tablet_manager.cc:1918] T fcdfe050b0e444b5ad7777a8eda2e029 P 65c21a86fd864c9da1cc0b52228a5132: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:34:52.223342 753 log.cc:1196] T fcdfe050b0e444b5ad7777a8eda2e029 P 65c21a86fd864c9da1cc0b52228a5132: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/ts-1/wal/wals/fcdfe050b0e444b5ad7777a8eda2e029
I20260806 06:34:52.223645 753 ts_tablet_manager.cc:1939] T fcdfe050b0e444b5ad7777a8eda2e029 P 65c21a86fd864c9da1cc0b52228a5132: Deleting consensus metadata
I20260806 06:34:52.224129 32459 catalog_manager.cc:4841] TS 65c21a86fd864c9da1cc0b52228a5132 (127.7.43.66:42149): tablet fcdfe050b0e444b5ad7777a8eda2e029 (table test-table [id=07b81d266b0649af81c092651c5dc1f8]) successfully deleted
I20260806 06:34:52.226419 755 tablet_replica.cc:330] T fcdfe050b0e444b5ad7777a8eda2e029 P 2d547e2d6d8946bea17fcdc9735ad8d3: stopping tablet replica
I20260806 06:34:52.226514 755 raft_consensus.cc:2229] T fcdfe050b0e444b5ad7777a8eda2e029 P 2d547e2d6d8946bea17fcdc9735ad8d3 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:34:52.226603 755 raft_consensus.cc:2258] T fcdfe050b0e444b5ad7777a8eda2e029 P 2d547e2d6d8946bea17fcdc9735ad8d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:34:52.226970 755 ts_tablet_manager.cc:1905] T fcdfe050b0e444b5ad7777a8eda2e029 P 2d547e2d6d8946bea17fcdc9735ad8d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:34:52.228432 755 ts_tablet_manager.cc:1918] T fcdfe050b0e444b5ad7777a8eda2e029 P 2d547e2d6d8946bea17fcdc9735ad8d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:34:52.228511 755 log.cc:1196] T fcdfe050b0e444b5ad7777a8eda2e029 P 2d547e2d6d8946bea17fcdc9735ad8d3: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785997861828271-7341-0/minicluster-data/ts-2/wal/wals/fcdfe050b0e444b5ad7777a8eda2e029
I20260806 06:34:52.228830 755 ts_tablet_manager.cc:1939] T fcdfe050b0e444b5ad7777a8eda2e029 P 2d547e2d6d8946bea17fcdc9735ad8d3: Deleting consensus metadata
I20260806 06:34:52.229194 32459 catalog_manager.cc:4841] TS 2d547e2d6d8946bea17fcdc9735ad8d3 (127.7.43.67:39975): tablet fcdfe050b0e444b5ad7777a8eda2e029 (table test-table [id=07b81d266b0649af81c092651c5dc1f8]) successfully deleted
W20260806 06:34:52.230532 32463 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.43.65:43505 (error 108)
I20260806 06:34:52.230968 708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:52.230845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:43505 (local address 127.0.0.1:60396)
0806 06:34:52.230895 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:52.230935 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.65:43505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 06:34:52.231091 32463 catalog_manager.cc:4584] TS b6f8a30db45947c790d3b13680095439 (127.7.43.65:43505): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fcdfe050b0e444b5ad7777a8eda2e029: Network error: Client connection negotiation failed: client connection to 127.7.43.65:43505: connect: Connection refused (error 111)
I20260806 06:34:52.231132 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 32764
I20260806 06:34:52.246771 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 503
I20260806 06:34:52.258312 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 32438
2026-08-06T06:34:52Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2022 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-06T06:34:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:34:52Z Disabled control of system clock
I20260806 06:34:52.286396 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:37383
--webserver_interface=127.7.43.126
--webserver_port=0
--builtin_ntp_servers=127.7.43.84:36657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.43.126:37383
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 06:34:52.364722 765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:34:52.364913 765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:34:52.364933 765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:34:52.365994 765 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:34:52.366027 765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:34:52.366039 765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:34:52.366050 765 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:34:52.366060 765 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:34:52.367364 765 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:36657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.43.126:37383
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:37383
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.43.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.765
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:34:52.367551 765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:34:52.367780 765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:34:52.372102 774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:34:52.372117 775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:34:52.372119 778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:34:52.372364 765 server_base.cc:1034] running on GCE node
I20260806 06:34:52.372615 765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:34:52.372879 765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:34:52.374014 765 hybrid_clock.cc:648] HybridClock initialized: now 1785998092374001 us; error 32 us; skew 500 ppm
I20260806 06:34:52.375116 765 webserver.cc:460] Webserver started at http://127.7.43.126:43233/ using document root <none> and password file <none>
I20260806 06:34:52.375331 765 fs_manager.cc:359] Metadata directory not provided
I20260806 06:34:52.375378 765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:34:52.375495 765 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:34:52.376441 765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/master-0/data/instance:
uuid: "379778abe2d24527a35d091c185e81da"
format_stamp: "Formatted at 2026-08-06 06:34:52 on dist-test-slave-6zbq"
server_key: "ac5b9f2264ffdb89e1d27688eadff7b1"
server_key_iv: "0ef8bebe75eeb77e214b1e1a05d70f8b"
server_key_version: "encryptionkey@0"
I20260806 06:34:52.376770 765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/master-0/wal/instance:
uuid: "379778abe2d24527a35d091c185e81da"
format_stamp: "Formatted at 2026-08-06 06:34:52 on dist-test-slave-6zbq"
server_key: "ac5b9f2264ffdb89e1d27688eadff7b1"
server_key_iv: "0ef8bebe75eeb77e214b1e1a05d70f8b"
server_key_version: "encryptionkey@0"
I20260806 06:34:52.377921 765 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:34:52.378571 786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:52.378726 765 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:34:52.378813 765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "379778abe2d24527a35d091c185e81da"
format_stamp: "Formatted at 2026-08-06 06:34:52 on dist-test-slave-6zbq"
server_key: "ac5b9f2264ffdb89e1d27688eadff7b1"
server_key_iv: "0ef8bebe75eeb77e214b1e1a05d70f8b"
server_key_version: "encryptionkey@0"
I20260806 06:34:52.378870 765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:34:52.425335 765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:34:52.425643 765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:34:52.425776 765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:34:52.429500 765 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.126:37383
I20260806 06:34:52.429535 857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.126:37383 every 8 connection(s)
I20260806 06:34:52.429845 765 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:34:52.430493 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 765
I20260806 06:34:52.430598 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/master-0/wal/instance
I20260806 06:34:52.430773 7341 external_mini_cluster.cc:1428] Setting key 8671b5084ed5f1a3cbf85ca2c0f5dd9b
I20260806 06:34:52.430847 859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:34:52.434070 859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 379778abe2d24527a35d091c185e81da: Bootstrap starting.
I20260806 06:34:52.434659 859 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 379778abe2d24527a35d091c185e81da: Neither blocks nor log segments found. Creating new log.
I20260806 06:34:52.434921 859 log.cc:824] T 00000000000000000000000000000000 P 379778abe2d24527a35d091c185e81da: Log is configured to *not* fsync() on all Append() calls
I20260806 06:34:52.435675 859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 379778abe2d24527a35d091c185e81da: No bootstrap required, opened a new log
I20260806 06:34:52.436255 862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:52.431263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53026 (local address 127.7.43.126:37383)
0806 06:34:52.431495 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:52.431504 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:52.431534 (+ 30us) server_negotiation.cc:407] Connection header received
0806 06:34:52.431927 (+ 393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:52.431936 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:52.432015 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:52.432157 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:34:52.432330 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.433614 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.434418 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.434869 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.434983 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:52.435626 (+ 643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:34:52.435630 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:34:52.435630 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:34:52.435803 (+ 173us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:34:52.435840 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:34:52.435851 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:34:52.435898 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:52.435991 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 06:34:52.436050 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":95,"threads_started":1}
I20260806 06:34:52.437618 859 raft_consensus.cc:348] T 00000000000000000000000000000000 P 379778abe2d24527a35d091c185e81da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "379778abe2d24527a35d091c185e81da" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 37383 } }
I20260806 06:34:52.437749 859 raft_consensus.cc:374] T 00000000000000000000000000000000 P 379778abe2d24527a35d091c185e81da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:34:52.437772 859 raft_consensus.cc:729] T 00000000000000000000000000000000 P 379778abe2d24527a35d091c185e81da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 379778abe2d24527a35d091c185e81da, State: Initialized, Role: FOLLOWER
I20260806 06:34:52.437855 859 consensus_queue.cc:260] T 00000000000000000000000000000000 P 379778abe2d24527a35d091c185e81da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "379778abe2d24527a35d091c185e81da" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 37383 } }
I20260806 06:34:52.437902 859 raft_consensus.cc:388] T 00000000000000000000000000000000 P 379778abe2d24527a35d091c185e81da [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:34:52.437927 859 raft_consensus.cc:482] T 00000000000000000000000000000000 P 379778abe2d24527a35d091c185e81da [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:34:52.437955 859 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 379778abe2d24527a35d091c185e81da [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:34:52.438452 859 raft_consensus.cc:504] T 00000000000000000000000000000000 P 379778abe2d24527a35d091c185e81da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "379778abe2d24527a35d091c185e81da" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 37383 } }
I20260806 06:34:52.438541 859 leader_election.cc:302] T 00000000000000000000000000000000 P 379778abe2d24527a35d091c185e81da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 379778abe2d24527a35d091c185e81da; no voters:
I20260806 06:34:52.438673 859 leader_election.cc:288] T 00000000000000000000000000000000 P 379778abe2d24527a35d091c185e81da [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:34:52.438717 864 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 379778abe2d24527a35d091c185e81da [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:34:52.438880 864 raft_consensus.cc:686] T 00000000000000000000000000000000 P 379778abe2d24527a35d091c185e81da [term 1 LEADER]: Becoming Leader. State: Replica: 379778abe2d24527a35d091c185e81da, State: Running, Role: LEADER
I20260806 06:34:52.438898 859 sys_catalog.cc:564] T 00000000000000000000000000000000 P 379778abe2d24527a35d091c185e81da [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:34:52.438966 864 consensus_queue.cc:237] T 00000000000000000000000000000000 P 379778abe2d24527a35d091c185e81da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "379778abe2d24527a35d091c185e81da" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 37383 } }
I20260806 06:34:52.439314 866 sys_catalog.cc:455] T 00000000000000000000000000000000 P 379778abe2d24527a35d091c185e81da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "379778abe2d24527a35d091c185e81da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "379778abe2d24527a35d091c185e81da" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 37383 } } }
I20260806 06:34:52.439405 866 sys_catalog.cc:458] T 00000000000000000000000000000000 P 379778abe2d24527a35d091c185e81da [sys.catalog]: This master's current role is: LEADER
I20260806 06:34:52.439566 867 sys_catalog.cc:455] T 00000000000000000000000000000000 P 379778abe2d24527a35d091c185e81da [sys.catalog]: SysCatalogTable state changed. Reason: New leader 379778abe2d24527a35d091c185e81da. Latest consensus state: current_term: 1 leader_uuid: "379778abe2d24527a35d091c185e81da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "379778abe2d24527a35d091c185e81da" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 37383 } } }
I20260806 06:34:52.439594 875 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:34:52.439620 867 sys_catalog.cc:458] T 00000000000000000000000000000000 P 379778abe2d24527a35d091c185e81da [sys.catalog]: This master's current role is: LEADER
I20260806 06:34:52.440037 875 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:34:52.441421 875 catalog_manager.cc:1347] Generated new cluster ID: cc0026ae4ce44db5839155d8be24ec41
I20260806 06:34:52.441462 875 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:34:52.451740 875 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:34:52.452225 875 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:34:52.458591 875 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 379778abe2d24527a35d091c185e81da: Generated new TSK 0
I20260806 06:34:52.458729 875 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:34:52.465268 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.65:0
--local_ip_for_outbound_sockets=127.7.43.65
--webserver_interface=127.7.43.65
--webserver_port=0
--tserver_master_addrs=127.7.43.126:37383
--builtin_ntp_servers=127.7.43.84:36657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 06:34:52.557695 885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:34:52.557886 885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:34:52.557915 885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:34:52.559124 885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:34:52.559175 885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:34:52.559209 885 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.65
I20260806 06:34:52.560545 885 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:36657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.43.65
--webserver_port=0
--tserver_master_addrs=127.7.43.126:37383
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.885
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.65
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:34:52.560783 885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:34:52.561039 885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:34:52.566747 896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:34:52.566805 893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:34:52.566762 892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:34:52.566877 885 server_base.cc:1034] running on GCE node
I20260806 06:34:52.567196 885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:34:52.567416 885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:34:52.568564 885 hybrid_clock.cc:648] HybridClock initialized: now 1785998092568547 us; error 30 us; skew 500 ppm
I20260806 06:34:52.569767 885 webserver.cc:460] Webserver started at http://127.7.43.65:38965/ using document root <none> and password file <none>
I20260806 06:34:52.569988 885 fs_manager.cc:359] Metadata directory not provided
I20260806 06:34:52.570034 885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:34:52.570168 885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:34:52.571231 885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/ts-0/data/instance:
uuid: "af28d7ae384346b8a2e55498db2d3dc2"
format_stamp: "Formatted at 2026-08-06 06:34:52 on dist-test-slave-6zbq"
server_key: "d46835d3e483ab19d25cc63b0255b28e"
server_key_iv: "9744de8c7fd7b3fe114225fb9289f703"
server_key_version: "encryptionkey@0"
I20260806 06:34:52.571556 885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/ts-0/wal/instance:
uuid: "af28d7ae384346b8a2e55498db2d3dc2"
format_stamp: "Formatted at 2026-08-06 06:34:52 on dist-test-slave-6zbq"
server_key: "d46835d3e483ab19d25cc63b0255b28e"
server_key_iv: "9744de8c7fd7b3fe114225fb9289f703"
server_key_version: "encryptionkey@0"
I20260806 06:34:52.572748 885 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260806 06:34:52.573432 904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:52.573573 885 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:34:52.573657 885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/ts-0/wal
uuid: "af28d7ae384346b8a2e55498db2d3dc2"
format_stamp: "Formatted at 2026-08-06 06:34:52 on dist-test-slave-6zbq"
server_key: "d46835d3e483ab19d25cc63b0255b28e"
server_key_iv: "9744de8c7fd7b3fe114225fb9289f703"
server_key_version: "encryptionkey@0"
I20260806 06:34:52.573721 885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:34:52.595836 885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:34:52.596074 885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:34:52.596187 885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:34:52.596374 885 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:34:52.596645 885 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:34:52.596676 885 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:52.596702 885 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:34:52.596726 885 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:52.602008 885 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.65:38133
I20260806 06:34:52.602046 1062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.65:38133 every 8 connection(s)
I20260806 06:34:52.602334 885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
I20260806 06:34:52.608420 862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:52.602996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:47585 (local address 127.7.43.126:37383)
0806 06:34:52.603074 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:52.603077 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:52.603263 (+ 186us) server_negotiation.cc:407] Connection header received
0806 06:34:52.603850 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:52.603852 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:52.603862 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:52.603901 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:34:52.604705 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.605447 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.606899 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.607142 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.607203 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:52.608039 (+ 836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:34:52.608041 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:34:52.608041 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:34:52.608046 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:34:52.608063 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:34:52.608068 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:34:52.608144 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:52.608256 (+ 112us) server_negotiation.cc:299] Negotiation successful
0806 06:34:52.608319 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 06:34:52.608494 1066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:52.602960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:37383 (local address 127.7.43.65:47585)
0806 06:34:52.603155 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:52.603185 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:52.603749 (+ 564us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:52.603972 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:52.603976 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:52.603986 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:34:52.604614 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:52.604622 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.605544 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.605545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:52.606820 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:52.606824 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.607228 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.607230 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:52.607384 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:52.607914 (+ 530us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:34:52.607914 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:34:52.607941 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:34:52.607962 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:34:52.608156 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:34:52.608173 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:34:52.608176 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:34:52.608231 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 06:34:52.608331 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":75,"threads_started":1}
I20260806 06:34:52.609078 1063 heartbeater.cc:347] Connected to a master server at 127.7.43.126:37383
I20260806 06:34:52.609175 1063 heartbeater.cc:464] Registering TS with master...
I20260806 06:34:52.609354 1063 heartbeater.cc:511] Master 127.7.43.126:37383 requested a full tablet report, sending...
I20260806 06:34:52.609905 807 ts_manager.cc:194] Registered new tserver with Master: af28d7ae384346b8a2e55498db2d3dc2 (127.7.43.65:38133)
I20260806 06:34:52.610796 807 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.43.65:47585
I20260806 06:34:52.611267 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 885
I20260806 06:34:52.611370 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/ts-0/wal/instance
I20260806 06:34:52.611543 7341 external_mini_cluster.cc:1428] Setting key fe421ff9cea98133f876ec11287f98a4
I20260806 06:34:52.612371 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.66:0
--local_ip_for_outbound_sockets=127.7.43.66
--webserver_interface=127.7.43.66
--webserver_port=0
--tserver_master_addrs=127.7.43.126:37383
--builtin_ntp_servers=127.7.43.84:36657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 06:34:52.692914 1068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:34:52.693099 1068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:34:52.693120 1068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:34:52.694267 1068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:34:52.694304 1068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:34:52.694329 1068 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.66
I20260806 06:34:52.695688 1068 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:36657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.43.66
--webserver_port=0
--tserver_master_addrs=127.7.43.126:37383
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1068
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.66
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:34:52.695885 1068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:34:52.696132 1068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:34:52.700605 1077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:34:52.700630 1074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:34:52.700700 1075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:34:52.700933 1068 server_base.cc:1034] running on GCE node
I20260806 06:34:52.701153 1068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:34:52.701360 1068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:34:52.702481 1068 hybrid_clock.cc:648] HybridClock initialized: now 1785998092702472 us; error 26 us; skew 500 ppm
I20260806 06:34:52.703598 1068 webserver.cc:460] Webserver started at http://127.7.43.66:40531/ using document root <none> and password file <none>
I20260806 06:34:52.703809 1068 fs_manager.cc:359] Metadata directory not provided
I20260806 06:34:52.703852 1068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:34:52.703976 1068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:34:52.704981 1068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/ts-1/data/instance:
uuid: "4ccddf494f66404681bad551fdc080f6"
format_stamp: "Formatted at 2026-08-06 06:34:52 on dist-test-slave-6zbq"
server_key: "f3eaa08728ee91956043decc1280205c"
server_key_iv: "39ef27deca6551ae279a6d63cad3e9fc"
server_key_version: "encryptionkey@0"
I20260806 06:34:52.705299 1068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/ts-1/wal/instance:
uuid: "4ccddf494f66404681bad551fdc080f6"
format_stamp: "Formatted at 2026-08-06 06:34:52 on dist-test-slave-6zbq"
server_key: "f3eaa08728ee91956043decc1280205c"
server_key_iv: "39ef27deca6551ae279a6d63cad3e9fc"
server_key_version: "encryptionkey@0"
I20260806 06:34:52.706600 1068 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 06:34:52.707397 1088 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:52.707522 1068 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:34:52.707604 1068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/ts-1/wal
uuid: "4ccddf494f66404681bad551fdc080f6"
format_stamp: "Formatted at 2026-08-06 06:34:52 on dist-test-slave-6zbq"
server_key: "f3eaa08728ee91956043decc1280205c"
server_key_iv: "39ef27deca6551ae279a6d63cad3e9fc"
server_key_version: "encryptionkey@0"
I20260806 06:34:52.707672 1068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:34:52.720664 1068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:34:52.720893 1068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:34:52.721001 1068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:34:52.721191 1068 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:34:52.721462 1068 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:34:52.721494 1068 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:52.721514 1068 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:34:52.721539 1068 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:52.726881 1068 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.66:40859
I20260806 06:34:52.726938 1254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.66:40859 every 8 connection(s)
I20260806 06:34:52.727219 1068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
I20260806 06:34:52.731884 862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:52.727840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:39663 (local address 127.7.43.126:37383)
0806 06:34:52.727887 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:52.727889 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:52.728078 (+ 189us) server_negotiation.cc:407] Connection header received
0806 06:34:52.728546 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:52.728548 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:52.728561 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:52.728611 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:34:52.729247 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.729724 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.730699 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.730897 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.730954 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:52.731621 (+ 667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:34:52.731622 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:34:52.731623 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:34:52.731628 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:34:52.731643 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:34:52.731648 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:34:52.731692 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:52.731776 (+ 84us) server_negotiation.cc:299] Negotiation successful
0806 06:34:52.731804 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:34:52.732043 1257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:52.727809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:37383 (local address 127.7.43.66:39663)
0806 06:34:52.727972 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:52.728001 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:52.728443 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:52.728675 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:52.728679 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:52.728689 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:34:52.729151 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:52.729156 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.729818 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.729819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:52.730614 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:52.730617 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.730967 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.730969 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:52.731089 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:52.731490 (+ 401us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:34:52.731491 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:34:52.731516 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:34:52.731536 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:34:52.731698 (+ 162us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:34:52.731704 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:34:52.731707 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:34:52.731764 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 06:34:52.731859 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":68,"threads_started":1}
I20260806 06:34:52.732497 1255 heartbeater.cc:347] Connected to a master server at 127.7.43.126:37383
I20260806 06:34:52.732585 1255 heartbeater.cc:464] Registering TS with master...
I20260806 06:34:52.732751 1255 heartbeater.cc:511] Master 127.7.43.126:37383 requested a full tablet report, sending...
I20260806 06:34:52.733114 807 ts_manager.cc:194] Registered new tserver with Master: 4ccddf494f66404681bad551fdc080f6 (127.7.43.66:40859)
I20260806 06:34:52.733453 807 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.43.66:39663
I20260806 06:34:52.737299 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 1068
I20260806 06:34:52.737391 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/ts-1/wal/instance
I20260806 06:34:52.737537 7341 external_mini_cluster.cc:1428] Setting key d9c08aad02c4bbbf4a69f4e638aa0a76
I20260806 06:34:52.738294 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.67:0
--local_ip_for_outbound_sockets=127.7.43.67
--webserver_interface=127.7.43.67
--webserver_port=0
--tserver_master_addrs=127.7.43.126:37383
--builtin_ntp_servers=127.7.43.84:36657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 06:34:52.817332 1258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:34:52.817503 1258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:34:52.817523 1258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:34:52.818603 1258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:34:52.818635 1258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:34:52.818658 1258 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.67
I20260806 06:34:52.819929 1258 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:36657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.43.67
--webserver_port=0
--tserver_master_addrs=127.7.43.126:37383
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1258
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.67
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:34:52.820103 1258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:34:52.820339 1258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:34:52.824769 1271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:34:52.824844 1258 server_base.cc:1034] running on GCE node
W20260806 06:34:52.824769 1268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:34:52.824826 1269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:34:52.825153 1258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:34:52.825344 1258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:34:52.826484 1258 hybrid_clock.cc:648] HybridClock initialized: now 1785998092826468 us; error 31 us; skew 500 ppm
I20260806 06:34:52.827621 1258 webserver.cc:460] Webserver started at http://127.7.43.67:33431/ using document root <none> and password file <none>
I20260806 06:34:52.827838 1258 fs_manager.cc:359] Metadata directory not provided
I20260806 06:34:52.827880 1258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:34:52.827996 1258 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:34:52.828949 1258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/ts-2/data/instance:
uuid: "24237ccf74e7464c899938849fdf94d3"
format_stamp: "Formatted at 2026-08-06 06:34:52 on dist-test-slave-6zbq"
server_key: "4260dd7d41bafe1449aac45b63f0b16d"
server_key_iv: "afa6a33c695f2de47bbe97cb142e86b3"
server_key_version: "encryptionkey@0"
I20260806 06:34:52.829250 1258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/ts-2/wal/instance:
uuid: "24237ccf74e7464c899938849fdf94d3"
format_stamp: "Formatted at 2026-08-06 06:34:52 on dist-test-slave-6zbq"
server_key: "4260dd7d41bafe1449aac45b63f0b16d"
server_key_iv: "afa6a33c695f2de47bbe97cb142e86b3"
server_key_version: "encryptionkey@0"
I20260806 06:34:52.830395 1258 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 06:34:52.831036 1280 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:52.831171 1258 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:34:52.831244 1258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/ts-2/wal
uuid: "24237ccf74e7464c899938849fdf94d3"
format_stamp: "Formatted at 2026-08-06 06:34:52 on dist-test-slave-6zbq"
server_key: "4260dd7d41bafe1449aac45b63f0b16d"
server_key_iv: "afa6a33c695f2de47bbe97cb142e86b3"
server_key_version: "encryptionkey@0"
I20260806 06:34:52.831311 1258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:34:52.840027 1258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:34:52.840333 1258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:34:52.840458 1258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:34:52.840641 1258 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:34:52.840922 1258 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:34:52.840955 1258 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:52.840974 1258 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:34:52.840997 1258 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:52.846403 1258 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.67:37045
I20260806 06:34:52.846465 1456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.67:37045 every 8 connection(s)
I20260806 06:34:52.846726 1258 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
I20260806 06:34:52.851243 862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:52.847332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:53151 (local address 127.7.43.126:37383)
0806 06:34:52.847375 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:52.847377 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:52.847571 (+ 194us) server_negotiation.cc:407] Connection header received
0806 06:34:52.848024 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:52.848025 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:52.848039 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:52.848091 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:34:52.848712 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.849190 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.850087 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.850278 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.850332 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:52.850994 (+ 662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:34:52.850995 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:34:52.850996 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:34:52.851000 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:34:52.851016 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:34:52.851020 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:34:52.851061 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:52.851136 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 06:34:52.851162 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:34:52.851425 1459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:52.847306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:37383 (local address 127.7.43.67:53151)
0806 06:34:52.847473 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:52.847501 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:52.847934 (+ 433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:52.848171 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:52.848175 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:52.848185 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:34:52.848626 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:52.848629 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.849267 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.849268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:52.850008 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:52.850010 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.850358 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.850360 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:52.850486 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:52.850874 (+ 388us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:34:52.850875 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:34:52.850899 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:34:52.850919 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:34:52.851067 (+ 148us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:34:52.851073 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:34:52.851075 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:34:52.851133 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 06:34:52.851248 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":77,"threads_started":1}
I20260806 06:34:52.851854 1457 heartbeater.cc:347] Connected to a master server at 127.7.43.126:37383
I20260806 06:34:52.851923 1457 heartbeater.cc:464] Registering TS with master...
I20260806 06:34:52.852077 1457 heartbeater.cc:511] Master 127.7.43.126:37383 requested a full tablet report, sending...
I20260806 06:34:52.852398 807 ts_manager.cc:194] Registered new tserver with Master: 24237ccf74e7464c899938849fdf94d3 (127.7.43.67:37045)
I20260806 06:34:52.852730 807 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.43.67:53151
I20260806 06:34:52.853123 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 1258
I20260806 06:34:52.853232 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/ts-2/wal/instance
I20260806 06:34:52.853404 7341 external_mini_cluster.cc:1428] Setting key 684af7576b90d43e6380ee7149da9b47
I20260806 06:34:52.854092 7341 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:34:52.857627 862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:52.855401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53036 (local address 127.7.43.126:37383)
0806 06:34:52.855435 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:52.855437 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:52.855517 (+ 80us) server_negotiation.cc:407] Connection header received
0806 06:34:52.855566 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:52.855567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:52.855574 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:52.855617 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:34:52.855739 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.856158 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.856805 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.856976 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.857022 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:52.857474 (+ 452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:34:52.857475 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:34:52.857476 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:34:52.857479 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:34:52.857490 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:34:52.857494 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:34:52.857529 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:52.857562 (+ 33us) server_negotiation.cc:299] Negotiation successful
0806 06:34:52.857578 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 06:34:52.858932 807 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53036:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:34:52.860388 807 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:34:52.866230 1471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:52.862322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44500 (local address 127.7.43.65:38133)
0806 06:34:52.862626 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:52.862631 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:52.862641 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:34:52.862715 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:52.862717 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:52.862733 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:52.862810 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:34:52.862986 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.864125 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.865357 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.865832 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.865897 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:52.865985 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:52.866108 (+ 123us) server_negotiation.cc:299] Negotiation successful
0806 06:34:52.866146 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":88,"threads_started":1}
I20260806 06:34:52.866288 1467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:52.862248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.67:37045 (local address 127.0.0.1:56262)
0806 06:34:52.862869 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:52.862876 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:52.862916 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:52.863103 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:52.863104 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:52.863106 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:34:52.863147 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:52.863148 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.863946 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.863947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:52.865137 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:52.865140 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.866048 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.866049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:52.866120 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:52.866125 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:34:52.866172 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 06:34:52.866213 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"mutex_wait_us":29,"thread_start_us":107,"threads_started":1}
I20260806 06:34:52.866401 1474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:52.862494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35820 (local address 127.7.43.66:40859)
0806 06:34:52.862684 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:52.862687 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:52.862797 (+ 110us) server_negotiation.cc:407] Connection header received
0806 06:34:52.862856 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:52.862858 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:52.862872 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:52.862954 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:34:52.863155 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.864202 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.865134 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.865862 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.865931 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:52.866011 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:52.866257 (+ 246us) server_negotiation.cc:299] Negotiation successful
0806 06:34:52.866299 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":74,"threads_started":1}
I20260806 06:34:52.866505 1473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:52.862447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56262 (local address 127.7.43.67:37045)
0806 06:34:52.862602 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:52.862607 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:52.862918 (+ 311us) server_negotiation.cc:407] Connection header received
0806 06:34:52.862969 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:52.862971 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:52.862984 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:52.863050 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:34:52.863219 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.863868 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.865221 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.865867 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.865931 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:52.866009 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:52.866402 (+ 393us) server_negotiation.cc:299] Negotiation successful
0806 06:34:52.866440 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":77,"threads_started":1}
I20260806 06:34:52.866555 1470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:52.862247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.66:40859 (local address 127.0.0.1:35820)
0806 06:34:52.862736 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:52.862745 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:52.862792 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:52.863048 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:52.863049 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:52.863051 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:34:52.863083 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:52.863084 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.864260 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.864260 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:52.865047 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:52.865050 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.866046 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.866047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:52.866128 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:52.866131 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:34:52.866182 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 06:34:52.866213 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"mutex_wait_us":25,"thread_start_us":109,"threads_started":1}
I20260806 06:34:52.866202 1468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:52.862233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:38133 (local address 127.0.0.1:44500)
0806 06:34:52.862518 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:52.862558 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:52.862622 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:52.862834 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:52.862837 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:52.862839 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:34:52.862903 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:52.862905 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.864206 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.864207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:52.865290 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:52.865291 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.865929 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.865931 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:52.866007 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:52.866020 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 06:34:52.866082 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 06:34:52.866130 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"mutex_wait_us":27,"thread_start_us":50,"threads_started":1}
I20260806 06:34:52.867179 1152 tablet_service.cc:1467] Processing CreateTablet for tablet fed9dfd1c1c048a7bdded8fdda02e2bb (DEFAULT_TABLE table=test-table [id=f6223fa778144d349cccb8c7f9d9e565]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:34:52.867218 973 tablet_service.cc:1467] Processing CreateTablet for tablet fed9dfd1c1c048a7bdded8fdda02e2bb (DEFAULT_TABLE table=test-table [id=f6223fa778144d349cccb8c7f9d9e565]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:34:52.867316 1348 tablet_service.cc:1467] Processing CreateTablet for tablet fed9dfd1c1c048a7bdded8fdda02e2bb (DEFAULT_TABLE table=test-table [id=f6223fa778144d349cccb8c7f9d9e565]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:34:52.867535 1152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fed9dfd1c1c048a7bdded8fdda02e2bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:34:52.867547 973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fed9dfd1c1c048a7bdded8fdda02e2bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:34:52.867558 1348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fed9dfd1c1c048a7bdded8fdda02e2bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:34:52.869815 1475 tablet_bootstrap.cc:492] T fed9dfd1c1c048a7bdded8fdda02e2bb P 4ccddf494f66404681bad551fdc080f6: Bootstrap starting.
I20260806 06:34:52.870579 1478 tablet_bootstrap.cc:492] T fed9dfd1c1c048a7bdded8fdda02e2bb P af28d7ae384346b8a2e55498db2d3dc2: Bootstrap starting.
I20260806 06:34:52.870702 1475 tablet_bootstrap.cc:654] T fed9dfd1c1c048a7bdded8fdda02e2bb P 4ccddf494f66404681bad551fdc080f6: Neither blocks nor log segments found. Creating new log.
I20260806 06:34:52.870784 1476 tablet_bootstrap.cc:492] T fed9dfd1c1c048a7bdded8fdda02e2bb P 24237ccf74e7464c899938849fdf94d3: Bootstrap starting.
I20260806 06:34:52.871023 1475 log.cc:824] T fed9dfd1c1c048a7bdded8fdda02e2bb P 4ccddf494f66404681bad551fdc080f6: Log is configured to *not* fsync() on all Append() calls
I20260806 06:34:52.871372 1476 tablet_bootstrap.cc:654] T fed9dfd1c1c048a7bdded8fdda02e2bb P 24237ccf74e7464c899938849fdf94d3: Neither blocks nor log segments found. Creating new log.
I20260806 06:34:52.871660 1476 log.cc:824] T fed9dfd1c1c048a7bdded8fdda02e2bb P 24237ccf74e7464c899938849fdf94d3: Log is configured to *not* fsync() on all Append() calls
I20260806 06:34:52.871765 1475 tablet_bootstrap.cc:492] T fed9dfd1c1c048a7bdded8fdda02e2bb P 4ccddf494f66404681bad551fdc080f6: No bootstrap required, opened a new log
I20260806 06:34:52.871833 1475 ts_tablet_manager.cc:1397] T fed9dfd1c1c048a7bdded8fdda02e2bb P 4ccddf494f66404681bad551fdc080f6: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 06:34:52.872272 1476 tablet_bootstrap.cc:492] T fed9dfd1c1c048a7bdded8fdda02e2bb P 24237ccf74e7464c899938849fdf94d3: No bootstrap required, opened a new log
I20260806 06:34:52.872330 1476 ts_tablet_manager.cc:1397] T fed9dfd1c1c048a7bdded8fdda02e2bb P 24237ccf74e7464c899938849fdf94d3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:34:52.872459 1478 tablet_bootstrap.cc:654] T fed9dfd1c1c048a7bdded8fdda02e2bb P af28d7ae384346b8a2e55498db2d3dc2: Neither blocks nor log segments found. Creating new log.
I20260806 06:34:52.872821 1478 log.cc:824] T fed9dfd1c1c048a7bdded8fdda02e2bb P af28d7ae384346b8a2e55498db2d3dc2: Log is configured to *not* fsync() on all Append() calls
I20260806 06:34:52.873579 1478 tablet_bootstrap.cc:492] T fed9dfd1c1c048a7bdded8fdda02e2bb P af28d7ae384346b8a2e55498db2d3dc2: No bootstrap required, opened a new log
I20260806 06:34:52.873680 1478 ts_tablet_manager.cc:1397] T fed9dfd1c1c048a7bdded8fdda02e2bb P af28d7ae384346b8a2e55498db2d3dc2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 06:34:52.874318 1475 raft_consensus.cc:348] T fed9dfd1c1c048a7bdded8fdda02e2bb P 4ccddf494f66404681bad551fdc080f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af28d7ae384346b8a2e55498db2d3dc2" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 38133 } } peers { permanent_uuid: "4ccddf494f66404681bad551fdc080f6" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40859 } } peers { permanent_uuid: "24237ccf74e7464c899938849fdf94d3" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 37045 } }
I20260806 06:34:52.874396 1476 raft_consensus.cc:348] T fed9dfd1c1c048a7bdded8fdda02e2bb P 24237ccf74e7464c899938849fdf94d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af28d7ae384346b8a2e55498db2d3dc2" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 38133 } } peers { permanent_uuid: "4ccddf494f66404681bad551fdc080f6" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40859 } } peers { permanent_uuid: "24237ccf74e7464c899938849fdf94d3" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 37045 } }
I20260806 06:34:52.874522 1476 raft_consensus.cc:374] T fed9dfd1c1c048a7bdded8fdda02e2bb P 24237ccf74e7464c899938849fdf94d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:34:52.874523 1475 raft_consensus.cc:374] T fed9dfd1c1c048a7bdded8fdda02e2bb P 4ccddf494f66404681bad551fdc080f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:34:52.874553 1476 raft_consensus.cc:729] T fed9dfd1c1c048a7bdded8fdda02e2bb P 24237ccf74e7464c899938849fdf94d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24237ccf74e7464c899938849fdf94d3, State: Initialized, Role: FOLLOWER
I20260806 06:34:52.874557 1475 raft_consensus.cc:729] T fed9dfd1c1c048a7bdded8fdda02e2bb P 4ccddf494f66404681bad551fdc080f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ccddf494f66404681bad551fdc080f6, State: Initialized, Role: FOLLOWER
I20260806 06:34:52.874696 1476 consensus_queue.cc:260] T fed9dfd1c1c048a7bdded8fdda02e2bb P 24237ccf74e7464c899938849fdf94d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af28d7ae384346b8a2e55498db2d3dc2" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 38133 } } peers { permanent_uuid: "4ccddf494f66404681bad551fdc080f6" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40859 } } peers { permanent_uuid: "24237ccf74e7464c899938849fdf94d3" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 37045 } }
I20260806 06:34:52.874694 1475 consensus_queue.cc:260] T fed9dfd1c1c048a7bdded8fdda02e2bb P 4ccddf494f66404681bad551fdc080f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af28d7ae384346b8a2e55498db2d3dc2" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 38133 } } peers { permanent_uuid: "4ccddf494f66404681bad551fdc080f6" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40859 } } peers { permanent_uuid: "24237ccf74e7464c899938849fdf94d3" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 37045 } }
I20260806 06:34:52.874939 1476 ts_tablet_manager.cc:1428] T fed9dfd1c1c048a7bdded8fdda02e2bb P 24237ccf74e7464c899938849fdf94d3: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 06:34:52.874943 1475 ts_tablet_manager.cc:1428] T fed9dfd1c1c048a7bdded8fdda02e2bb P 4ccddf494f66404681bad551fdc080f6: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 06:34:52.875039 1457 heartbeater.cc:503] Master 127.7.43.126:37383 was elected leader, sending a full tablet report...
I20260806 06:34:52.875046 1255 heartbeater.cc:503] Master 127.7.43.126:37383 was elected leader, sending a full tablet report...
I20260806 06:34:52.876070 1478 raft_consensus.cc:348] T fed9dfd1c1c048a7bdded8fdda02e2bb P af28d7ae384346b8a2e55498db2d3dc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af28d7ae384346b8a2e55498db2d3dc2" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 38133 } } peers { permanent_uuid: "4ccddf494f66404681bad551fdc080f6" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40859 } } peers { permanent_uuid: "24237ccf74e7464c899938849fdf94d3" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 37045 } }
I20260806 06:34:52.876214 1478 raft_consensus.cc:374] T fed9dfd1c1c048a7bdded8fdda02e2bb P af28d7ae384346b8a2e55498db2d3dc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:34:52.876243 1478 raft_consensus.cc:729] T fed9dfd1c1c048a7bdded8fdda02e2bb P af28d7ae384346b8a2e55498db2d3dc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af28d7ae384346b8a2e55498db2d3dc2, State: Initialized, Role: FOLLOWER
I20260806 06:34:52.876322 1478 consensus_queue.cc:260] T fed9dfd1c1c048a7bdded8fdda02e2bb P af28d7ae384346b8a2e55498db2d3dc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af28d7ae384346b8a2e55498db2d3dc2" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 38133 } } peers { permanent_uuid: "4ccddf494f66404681bad551fdc080f6" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40859 } } peers { permanent_uuid: "24237ccf74e7464c899938849fdf94d3" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 37045 } }
I20260806 06:34:52.876528 1478 ts_tablet_manager.cc:1428] T fed9dfd1c1c048a7bdded8fdda02e2bb P af28d7ae384346b8a2e55498db2d3dc2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:34:52.876586 1063 heartbeater.cc:503] Master 127.7.43.126:37383 was elected leader, sending a full tablet report...
I20260806 06:34:52.925973 1486 raft_consensus.cc:482] T fed9dfd1c1c048a7bdded8fdda02e2bb P 24237ccf74e7464c899938849fdf94d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:34:52.926082 1486 raft_consensus.cc:504] T fed9dfd1c1c048a7bdded8fdda02e2bb P 24237ccf74e7464c899938849fdf94d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af28d7ae384346b8a2e55498db2d3dc2" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 38133 } } peers { permanent_uuid: "4ccddf494f66404681bad551fdc080f6" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40859 } } peers { permanent_uuid: "24237ccf74e7464c899938849fdf94d3" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 37045 } }
I20260806 06:34:52.926442 1486 leader_election.cc:288] T fed9dfd1c1c048a7bdded8fdda02e2bb P 24237ccf74e7464c899938849fdf94d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers af28d7ae384346b8a2e55498db2d3dc2 (127.7.43.65:38133), 4ccddf494f66404681bad551fdc080f6 (127.7.43.66:40859)
I20260806 06:34:52.928988 1474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:52.926664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:38687 (local address 127.7.43.66:40859)
0806 06:34:52.926750 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:52.926752 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:52.926759 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:34:52.926784 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:52.926785 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:52.926792 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:52.926835 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:34:52.926965 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.927436 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.928351 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.928702 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.928760 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:52.928770 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:52.928891 (+ 121us) server_negotiation.cc:299] Negotiation successful
0806 06:34:52.928923 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 06:34:52.928990 1459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:52.926572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.66:40859 (local address 127.7.43.67:38687)
0806 06:34:52.926647 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:52.926658 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:52.926707 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:52.926848 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:52.926848 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:52.926849 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:34:52.926912 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:52.926913 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.927509 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.927510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:52.928271 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:52.928273 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.928774 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.928775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:52.928834 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:52.928837 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:34:52.928880 (+ 43us) client_negotiation.cc:240] Negotiation successful
0806 06:34:52.928921 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57,"mutex_wait_us":33}
I20260806 06:34:52.929081 1471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:52.926711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:33357 (local address 127.7.43.65:38133)
0806 06:34:52.926767 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:52.926769 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:52.926810 (+ 41us) server_negotiation.cc:407] Connection header received
0806 06:34:52.926872 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:52.926873 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:52.926885 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:52.926932 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:34:52.927071 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.927542 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.928428 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.928786 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.928844 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:52.928854 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:52.928988 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 06:34:52.929022 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 06:34:52.929095 1489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:52.926575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:38133 (local address 127.7.43.67:33357)
0806 06:34:52.926754 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:52.926766 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:52.926826 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:52.926943 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:52.926944 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:52.926945 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:34:52.927014 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:52.927015 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.927615 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.927616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:52.928351 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:52.928352 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.928858 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.928859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:52.928931 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:52.928934 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:34:52.928976 (+ 42us) client_negotiation.cc:240] Negotiation successful
0806 06:34:52.929011 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":11,"thread_start_us":114,"threads_started":1}
I20260806 06:34:52.929352 1176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fed9dfd1c1c048a7bdded8fdda02e2bb" candidate_uuid: "24237ccf74e7464c899938849fdf94d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ccddf494f66404681bad551fdc080f6" is_pre_election: true
I20260806 06:34:52.929498 1176 raft_consensus.cc:2454] T fed9dfd1c1c048a7bdded8fdda02e2bb P 4ccddf494f66404681bad551fdc080f6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24237ccf74e7464c899938849fdf94d3 in term 0.
I20260806 06:34:52.929472 1002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fed9dfd1c1c048a7bdded8fdda02e2bb" candidate_uuid: "24237ccf74e7464c899938849fdf94d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af28d7ae384346b8a2e55498db2d3dc2" is_pre_election: true
I20260806 06:34:52.929606 1002 raft_consensus.cc:2454] T fed9dfd1c1c048a7bdded8fdda02e2bb P af28d7ae384346b8a2e55498db2d3dc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24237ccf74e7464c899938849fdf94d3 in term 0.
I20260806 06:34:52.929723 1281 leader_election.cc:302] T fed9dfd1c1c048a7bdded8fdda02e2bb P 24237ccf74e7464c899938849fdf94d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24237ccf74e7464c899938849fdf94d3, 4ccddf494f66404681bad551fdc080f6; no voters:
I20260806 06:34:52.929842 1486 raft_consensus.cc:2781] T fed9dfd1c1c048a7bdded8fdda02e2bb P 24237ccf74e7464c899938849fdf94d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:34:52.929895 1486 raft_consensus.cc:482] T fed9dfd1c1c048a7bdded8fdda02e2bb P 24237ccf74e7464c899938849fdf94d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:34:52.929922 1486 raft_consensus.cc:3037] T fed9dfd1c1c048a7bdded8fdda02e2bb P 24237ccf74e7464c899938849fdf94d3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:34:52.930581 1486 raft_consensus.cc:504] T fed9dfd1c1c048a7bdded8fdda02e2bb P 24237ccf74e7464c899938849fdf94d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af28d7ae384346b8a2e55498db2d3dc2" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 38133 } } peers { permanent_uuid: "4ccddf494f66404681bad551fdc080f6" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40859 } } peers { permanent_uuid: "24237ccf74e7464c899938849fdf94d3" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 37045 } }
I20260806 06:34:52.930694 1486 leader_election.cc:288] T fed9dfd1c1c048a7bdded8fdda02e2bb P 24237ccf74e7464c899938849fdf94d3 [CANDIDATE]: Term 1 election: Requested vote from peers af28d7ae384346b8a2e55498db2d3dc2 (127.7.43.65:38133), 4ccddf494f66404681bad551fdc080f6 (127.7.43.66:40859)
I20260806 06:34:52.930815 1002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fed9dfd1c1c048a7bdded8fdda02e2bb" candidate_uuid: "24237ccf74e7464c899938849fdf94d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af28d7ae384346b8a2e55498db2d3dc2"
I20260806 06:34:52.930842 1176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fed9dfd1c1c048a7bdded8fdda02e2bb" candidate_uuid: "24237ccf74e7464c899938849fdf94d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ccddf494f66404681bad551fdc080f6"
I20260806 06:34:52.930896 1176 raft_consensus.cc:3037] T fed9dfd1c1c048a7bdded8fdda02e2bb P 4ccddf494f66404681bad551fdc080f6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:34:52.930882 1002 raft_consensus.cc:3037] T fed9dfd1c1c048a7bdded8fdda02e2bb P af28d7ae384346b8a2e55498db2d3dc2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:34:52.931550 1176 raft_consensus.cc:2454] T fed9dfd1c1c048a7bdded8fdda02e2bb P 4ccddf494f66404681bad551fdc080f6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24237ccf74e7464c899938849fdf94d3 in term 1.
I20260806 06:34:52.931581 1002 raft_consensus.cc:2454] T fed9dfd1c1c048a7bdded8fdda02e2bb P af28d7ae384346b8a2e55498db2d3dc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24237ccf74e7464c899938849fdf94d3 in term 1.
I20260806 06:34:52.931713 1281 leader_election.cc:302] T fed9dfd1c1c048a7bdded8fdda02e2bb P 24237ccf74e7464c899938849fdf94d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24237ccf74e7464c899938849fdf94d3, 4ccddf494f66404681bad551fdc080f6; no voters:
I20260806 06:34:52.931807 1486 raft_consensus.cc:2781] T fed9dfd1c1c048a7bdded8fdda02e2bb P 24237ccf74e7464c899938849fdf94d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:34:52.931951 1486 raft_consensus.cc:686] T fed9dfd1c1c048a7bdded8fdda02e2bb P 24237ccf74e7464c899938849fdf94d3 [term 1 LEADER]: Becoming Leader. State: Replica: 24237ccf74e7464c899938849fdf94d3, State: Running, Role: LEADER
I20260806 06:34:52.932035 1486 consensus_queue.cc:237] T fed9dfd1c1c048a7bdded8fdda02e2bb P 24237ccf74e7464c899938849fdf94d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af28d7ae384346b8a2e55498db2d3dc2" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 38133 } } peers { permanent_uuid: "4ccddf494f66404681bad551fdc080f6" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40859 } } peers { permanent_uuid: "24237ccf74e7464c899938849fdf94d3" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 37045 } }
I20260806 06:34:52.932811 806 catalog_manager.cc:5495] T fed9dfd1c1c048a7bdded8fdda02e2bb P 24237ccf74e7464c899938849fdf94d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 24237ccf74e7464c899938849fdf94d3 (127.7.43.67). New cstate: current_term: 1 leader_uuid: "24237ccf74e7464c899938849fdf94d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af28d7ae384346b8a2e55498db2d3dc2" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 38133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ccddf494f66404681bad551fdc080f6" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24237ccf74e7464c899938849fdf94d3" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 37045 } health_report { overall_health: HEALTHY } } }
I20260806 06:34:52.955679 1473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:52.953621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56276 (local address 127.7.43.67:37045)
0806 06:34:52.953672 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:52.953674 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:52.953682 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:34:52.953717 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:52.953718 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:52.953731 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:52.953786 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:34:52.953872 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.954322 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.955028 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:52.955204 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:52.955250 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:52.955379 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:34:52.955471 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:34:52.955565 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:52.955582 (+ 17us) server_negotiation.cc:299] Negotiation successful
0806 06:34:52.955599 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 06:34:52.957158 1002 raft_consensus.cc:1261] T fed9dfd1c1c048a7bdded8fdda02e2bb P af28d7ae384346b8a2e55498db2d3dc2 [term 1 FOLLOWER]: Refusing update from remote peer 24237ccf74e7464c899938849fdf94d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:34:52.957157 1176 raft_consensus.cc:1261] T fed9dfd1c1c048a7bdded8fdda02e2bb P 4ccddf494f66404681bad551fdc080f6 [term 1 FOLLOWER]: Refusing update from remote peer 24237ccf74e7464c899938849fdf94d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:34:52.957472 1486 consensus_queue.cc:1035] T fed9dfd1c1c048a7bdded8fdda02e2bb P 24237ccf74e7464c899938849fdf94d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af28d7ae384346b8a2e55498db2d3dc2" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 38133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:34:52.957598 1490 consensus_queue.cc:1035] T fed9dfd1c1c048a7bdded8fdda02e2bb P 24237ccf74e7464c899938849fdf94d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ccddf494f66404681bad551fdc080f6" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 40859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:34:52.959647 1495 mvcc.cc:205] Tried to move back new op lower bound from 7315448188750532608 to 7315448188650295296. Current Snapshot: MvccSnapshot[applied={T|T < 7315448188750532608}]
I20260806 06:34:52.960577 1494 mvcc.cc:205] Tried to move back new op lower bound from 7315448188750532608 to 7315448188650295296. Current Snapshot: MvccSnapshot[applied={T|T < 7315448188750532608}]
I20260806 06:34:52.961585 807 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53036:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:34:52.961658 807 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53036:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:34:52.962287 807 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 379778abe2d24527a35d091c185e81da: Sending DeleteTablet for 3 replicas of tablet fed9dfd1c1c048a7bdded8fdda02e2bb
I20260806 06:34:52.962591 1152 tablet_service.cc:1514] Processing DeleteTablet for tablet fed9dfd1c1c048a7bdded8fdda02e2bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:34:52 UTC) from {username='slave'} at 127.0.0.1:35820
I20260806 06:34:52.962638 973 tablet_service.cc:1514] Processing DeleteTablet for tablet fed9dfd1c1c048a7bdded8fdda02e2bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:34:52 UTC) from {username='slave'} at 127.0.0.1:44500
I20260806 06:34:52.962739 1348 tablet_service.cc:1514] Processing DeleteTablet for tablet fed9dfd1c1c048a7bdded8fdda02e2bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:34:52 UTC) from {username='slave'} at 127.0.0.1:56262
I20260806 06:34:52.962875 1507 tablet_replica.cc:330] T fed9dfd1c1c048a7bdded8fdda02e2bb P af28d7ae384346b8a2e55498db2d3dc2: stopping tablet replica
I20260806 06:34:52.962924 1508 tablet_replica.cc:330] T fed9dfd1c1c048a7bdded8fdda02e2bb P 24237ccf74e7464c899938849fdf94d3: stopping tablet replica
I20260806 06:34:52.962987 1506 tablet_replica.cc:330] T fed9dfd1c1c048a7bdded8fdda02e2bb P 4ccddf494f66404681bad551fdc080f6: stopping tablet replica
I20260806 06:34:52.962998 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 885
I20260806 06:34:52.963059 1508 raft_consensus.cc:2229] T fed9dfd1c1c048a7bdded8fdda02e2bb P 24237ccf74e7464c899938849fdf94d3 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:34:52.963152 1508 raft_consensus.cc:2258] T fed9dfd1c1c048a7bdded8fdda02e2bb P 24237ccf74e7464c899938849fdf94d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:34:52.965193 1506 raft_consensus.cc:2229] T fed9dfd1c1c048a7bdded8fdda02e2bb P 4ccddf494f66404681bad551fdc080f6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:34:52.966454 1506 raft_consensus.cc:2258] T fed9dfd1c1c048a7bdded8fdda02e2bb P 4ccddf494f66404681bad551fdc080f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:34:52.970306 1506 ts_tablet_manager.cc:1905] T fed9dfd1c1c048a7bdded8fdda02e2bb P 4ccddf494f66404681bad551fdc080f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:34:52.971550 1508 ts_tablet_manager.cc:1905] T fed9dfd1c1c048a7bdded8fdda02e2bb P 24237ccf74e7464c899938849fdf94d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:34:52.971675 1506 ts_tablet_manager.cc:1918] T fed9dfd1c1c048a7bdded8fdda02e2bb P 4ccddf494f66404681bad551fdc080f6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:34:52.971794 1506 log.cc:1196] T fed9dfd1c1c048a7bdded8fdda02e2bb P 4ccddf494f66404681bad551fdc080f6: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/ts-1/wal/wals/fed9dfd1c1c048a7bdded8fdda02e2bb
I20260806 06:34:52.972086 1506 ts_tablet_manager.cc:1939] T fed9dfd1c1c048a7bdded8fdda02e2bb P 4ccddf494f66404681bad551fdc080f6: Deleting consensus metadata
I20260806 06:34:52.972546 787 catalog_manager.cc:4841] TS 4ccddf494f66404681bad551fdc080f6 (127.7.43.66:40859): tablet fed9dfd1c1c048a7bdded8fdda02e2bb (table test-table [id=f6223fa778144d349cccb8c7f9d9e565]) successfully deleted
I20260806 06:34:52.973040 1508 ts_tablet_manager.cc:1918] T fed9dfd1c1c048a7bdded8fdda02e2bb P 24237ccf74e7464c899938849fdf94d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:34:52.973107 1508 log.cc:1196] T fed9dfd1c1c048a7bdded8fdda02e2bb P 24237ccf74e7464c899938849fdf94d3: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785997861828271-7341-0/minicluster-data/ts-2/wal/wals/fed9dfd1c1c048a7bdded8fdda02e2bb
I20260806 06:34:52.973309 1508 ts_tablet_manager.cc:1939] T fed9dfd1c1c048a7bdded8fdda02e2bb P 24237ccf74e7464c899938849fdf94d3: Deleting consensus metadata
I20260806 06:34:52.973646 790 catalog_manager.cc:4841] TS 24237ccf74e7464c899938849fdf94d3 (127.7.43.67:37045): tablet fed9dfd1c1c048a7bdded8fdda02e2bb (table test-table [id=f6223fa778144d349cccb8c7f9d9e565]) successfully deleted
W20260806 06:34:52.976351 789 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.43.65:38133 (error 108)
I20260806 06:34:52.976773 1468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:52.976658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:38133 (local address 127.0.0.1:44512)
0806 06:34:52.976702 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:52.976740 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.65:38133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260806 06:34:52.976887 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 1068
W20260806 06:34:52.976931 789 catalog_manager.cc:4584] TS af28d7ae384346b8a2e55498db2d3dc2 (127.7.43.65:38133): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fed9dfd1c1c048a7bdded8fdda02e2bb: Network error: Client connection negotiation failed: client connection to 127.7.43.65:38133: connect: Connection refused (error 111)
I20260806 06:34:52.984725 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 1258
I20260806 06:34:52.996937 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 765
2026-08-06T06:34:52Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (738 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:34:53 dist-test-slave-6zbq krb5kdc[1517](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:34:53 dist-test-slave-6zbq krb5kdc[1517](info): set up 2 sockets
Aug 06 06:34:53 dist-test-slave-6zbq krb5kdc[1517](info): commencing operation
krb5kdc: starting...
W20260806 06:34:54.090689 7341 mini_kdc.cc:121] Time spent starting KDC: real 1.073s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Aug 06 06:34:54 dist-test-slave-6zbq krb5kdc[1517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998094, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:34:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:34:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.43.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
I20260806 06:34:54.208669 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:43743
--webserver_interface=127.7.43.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:37381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.43.126:43743
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:34:54.290596 1538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:34:54.290774 1538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:34:54.290795 1538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:34:54.291872 1538 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:34:54.291911 1538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:34:54.291924 1538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:34:54.291935 1538 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:34:54.291958 1538 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:34:54.293377 1538 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:37381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.43.126:43743
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:43743
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.43.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1538
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:34:54.293587 1538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:34:54.293772 1538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:34:54.298288 1552 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:34:54.298290 1550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:34:54.298373 1548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:34:54.298427 1538 server_base.cc:1034] running on GCE node
I20260806 06:34:54.298731 1538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:34:54.298992 1538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:34:54.300134 1538 hybrid_clock.cc:648] HybridClock initialized: now 1785998094300114 us; error 26 us; skew 500 ppm
Aug 06 06:34:54 dist-test-slave-6zbq krb5kdc[1517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998094, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:34:54.308768 1538 init.cc:377] Logged in from keytab as kudu/127.7.43.126@KRBTEST.COM (short username kudu)
I20260806 06:34:54.309166 1538 webserver.cc:460] Webserver started at http://127.7.43.126:45187/ using document root <none> and password file <none>
I20260806 06:34:54.309347 1538 fs_manager.cc:359] Metadata directory not provided
I20260806 06:34:54.309386 1538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:34:54.309476 1538 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:34:54.310485 1538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/master-0/data/instance:
uuid: "ad0fabc5ddf645319a6fcb30fbe2f6e3"
format_stamp: "Formatted at 2026-08-06 06:34:54 on dist-test-slave-6zbq"
server_key: "275be2c5a6748f44738457362378a944"
server_key_iv: "7fa823a1ca7eb6351acb8cdd7a52e4ea"
server_key_version: "encryptionkey@0"
I20260806 06:34:54.310773 1538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/master-0/wal/instance:
uuid: "ad0fabc5ddf645319a6fcb30fbe2f6e3"
format_stamp: "Formatted at 2026-08-06 06:34:54 on dist-test-slave-6zbq"
server_key: "275be2c5a6748f44738457362378a944"
server_key_iv: "7fa823a1ca7eb6351acb8cdd7a52e4ea"
server_key_version: "encryptionkey@0"
I20260806 06:34:54.311909 1538 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:34:54.312521 1561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:54.312690 1538 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:34:54.312770 1538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "ad0fabc5ddf645319a6fcb30fbe2f6e3"
format_stamp: "Formatted at 2026-08-06 06:34:54 on dist-test-slave-6zbq"
server_key: "275be2c5a6748f44738457362378a944"
server_key_iv: "7fa823a1ca7eb6351acb8cdd7a52e4ea"
server_key_version: "encryptionkey@0"
I20260806 06:34:54.312824 1538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:34:54.326220 1538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:34:54.328092 1538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:34:54.328202 1538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:34:54.331740 1538 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.126:43743
I20260806 06:34:54.331791 1636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.126:43743 every 8 connection(s)
I20260806 06:34:54.332067 1538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:34:54.332846 1637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:34:54.333743 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 1538
I20260806 06:34:54.333844 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/master-0/wal/instance
I20260806 06:34:54.334021 7341 external_mini_cluster.cc:1428] Setting key 0d71c8ef8c5ea56e59ae7d1c0952836e
I20260806 06:34:54.335388 1637 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad0fabc5ddf645319a6fcb30fbe2f6e3: Bootstrap starting.
I20260806 06:34:54.335937 1637 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad0fabc5ddf645319a6fcb30fbe2f6e3: Neither blocks nor log segments found. Creating new log.
I20260806 06:34:54.336238 1637 log.cc:824] T 00000000000000000000000000000000 P ad0fabc5ddf645319a6fcb30fbe2f6e3: Log is configured to *not* fsync() on all Append() calls
I20260806 06:34:54.336937 1637 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad0fabc5ddf645319a6fcb30fbe2f6e3: No bootstrap required, opened a new log
I20260806 06:34:54.338974 1637 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad0fabc5ddf645319a6fcb30fbe2f6e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad0fabc5ddf645319a6fcb30fbe2f6e3" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 43743 } }
I20260806 06:34:54.339097 1637 raft_consensus.cc:374] T 00000000000000000000000000000000 P ad0fabc5ddf645319a6fcb30fbe2f6e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:34:54.339118 1637 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad0fabc5ddf645319a6fcb30fbe2f6e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad0fabc5ddf645319a6fcb30fbe2f6e3, State: Initialized, Role: FOLLOWER
I20260806 06:34:54.339222 1637 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad0fabc5ddf645319a6fcb30fbe2f6e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad0fabc5ddf645319a6fcb30fbe2f6e3" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 43743 } }
I20260806 06:34:54.339290 1637 raft_consensus.cc:388] T 00000000000000000000000000000000 P ad0fabc5ddf645319a6fcb30fbe2f6e3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:34:54.339313 1637 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad0fabc5ddf645319a6fcb30fbe2f6e3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:34:54.339357 1637 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad0fabc5ddf645319a6fcb30fbe2f6e3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:34:54.339838 1637 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad0fabc5ddf645319a6fcb30fbe2f6e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad0fabc5ddf645319a6fcb30fbe2f6e3" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 43743 } }
I20260806 06:34:54.339920 1637 leader_election.cc:302] T 00000000000000000000000000000000 P ad0fabc5ddf645319a6fcb30fbe2f6e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad0fabc5ddf645319a6fcb30fbe2f6e3; no voters:
I20260806 06:34:54.340037 1637 leader_election.cc:288] T 00000000000000000000000000000000 P ad0fabc5ddf645319a6fcb30fbe2f6e3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:34:54.340095 1647 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad0fabc5ddf645319a6fcb30fbe2f6e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:34:54.340199 1647 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad0fabc5ddf645319a6fcb30fbe2f6e3 [term 1 LEADER]: Becoming Leader. State: Replica: ad0fabc5ddf645319a6fcb30fbe2f6e3, State: Running, Role: LEADER
I20260806 06:34:54.340270 1637 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad0fabc5ddf645319a6fcb30fbe2f6e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:34:54.340279 1647 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad0fabc5ddf645319a6fcb30fbe2f6e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad0fabc5ddf645319a6fcb30fbe2f6e3" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 43743 } }
Aug 06 06:34:54 dist-test-slave-6zbq krb5kdc[1517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998094, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:34:54.340682 1648 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad0fabc5ddf645319a6fcb30fbe2f6e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ad0fabc5ddf645319a6fcb30fbe2f6e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad0fabc5ddf645319a6fcb30fbe2f6e3" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 43743 } } }
I20260806 06:34:54.340758 1648 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad0fabc5ddf645319a6fcb30fbe2f6e3 [sys.catalog]: This master's current role is: LEADER
I20260806 06:34:54.340967 1649 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad0fabc5ddf645319a6fcb30fbe2f6e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad0fabc5ddf645319a6fcb30fbe2f6e3. Latest consensus state: current_term: 1 leader_uuid: "ad0fabc5ddf645319a6fcb30fbe2f6e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad0fabc5ddf645319a6fcb30fbe2f6e3" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 43743 } } }
I20260806 06:34:54.341055 1649 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad0fabc5ddf645319a6fcb30fbe2f6e3 [sys.catalog]: This master's current role is: LEADER
W20260806 06:34:54.342082 1667 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ad0fabc5ddf645319a6fcb30fbe2f6e3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:34:54.342315 1667 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:34:54.342429 1659 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:34:54.342582 1659 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:34:54.344010 1659 catalog_manager.cc:1347] Generated new cluster ID: 3b589ffa2a194dce9aa4a12bb9973a25
I20260806 06:34:54.344063 1659 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:34:54.344605 1644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:54.334491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44156 (local address 127.7.43.126:43743)
0806 06:34:54.334657 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:54.334666 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:54.334698 (+ 32us) server_negotiation.cc:407] Connection header received
0806 06:34:54.335161 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:54.335170 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:54.335236 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:54.335415 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:34:54.335995 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:54.337225 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:54.337943 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:54.338609 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:54.338720 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:54.340948 (+ 2228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:34:54.340952 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:34:54.340952 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:34:54.340963 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:34:54.343006 (+ 2043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:34:54.343339 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:34:54.343342 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:34:54.343343 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:34:54.343462 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:34:54.343689 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:34:54.343691 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:34:54.343691 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:34:54.343910 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:34:54.344048 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:54.344297 (+ 249us) server_negotiation.cc:299] Negotiation successful
0806 06:34:54.344423 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":75,"threads_started":1}
I20260806 06:34:54.366442 1659 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:34:54.367030 1659 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:34:54.370103 1659 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ad0fabc5ddf645319a6fcb30fbe2f6e3: Generated new TSK 0
I20260806 06:34:54.370247 1659 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.43.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab.
Entry for principal kudu/127.7.43.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab.
I20260806 06:34:54.400362 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.65:0
--local_ip_for_outbound_sockets=127.7.43.65
--webserver_interface=127.7.43.65
--webserver_port=0
--tserver_master_addrs=127.7.43.126:43743
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab
--principal=kudu/127.7.43.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:37381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:34:54.482246 1677 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:34:54.482442 1677 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:34:54.482465 1677 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:34:54.483530 1677 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:34:54.483563 1677 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:34:54.483588 1677 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.65
I20260806 06:34:54.484984 1677 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:37381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab
--principal=kudu/127.7.43.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.43.65
--webserver_port=0
--tserver_master_addrs=127.7.43.126:43743
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1677
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.65
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:34:54.485158 1677 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:34:54.485319 1677 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:34:54.489787 1687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:34:54.489823 1677 server_base.cc:1034] running on GCE node
W20260806 06:34:54.489792 1683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:34:54.489799 1685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:34:54.490150 1677 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:34:54.490406 1677 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:34:54.491537 1677 hybrid_clock.cc:648] HybridClock initialized: now 1785998094491523 us; error 31 us; skew 500 ppm
Aug 06 06:34:54 dist-test-slave-6zbq krb5kdc[1517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998094, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:34:54.493781 1677 init.cc:377] Logged in from keytab as kudu/127.7.43.65@KRBTEST.COM (short username kudu)
I20260806 06:34:54.494151 1677 webserver.cc:460] Webserver started at http://127.7.43.65:43969/ using document root <none> and password file <none>
I20260806 06:34:54.494374 1677 fs_manager.cc:359] Metadata directory not provided
I20260806 06:34:54.494426 1677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:34:54.494542 1677 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:34:54.495476 1677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/ts-0/data/instance:
uuid: "04c42cac64f54ff98de03eef3df52b74"
format_stamp: "Formatted at 2026-08-06 06:34:54 on dist-test-slave-6zbq"
server_key: "f1e1a6acb5de0f9d5885149df99fb923"
server_key_iv: "8b037745e7563c06545bf1c5f1f9f4a9"
server_key_version: "encryptionkey@0"
I20260806 06:34:54.495790 1677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/ts-0/wal/instance:
uuid: "04c42cac64f54ff98de03eef3df52b74"
format_stamp: "Formatted at 2026-08-06 06:34:54 on dist-test-slave-6zbq"
server_key: "f1e1a6acb5de0f9d5885149df99fb923"
server_key_iv: "8b037745e7563c06545bf1c5f1f9f4a9"
server_key_version: "encryptionkey@0"
I20260806 06:34:54.496960 1677 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:34:54.497637 1695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:54.497787 1677 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:34:54.497866 1677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/ts-0/wal
uuid: "04c42cac64f54ff98de03eef3df52b74"
format_stamp: "Formatted at 2026-08-06 06:34:54 on dist-test-slave-6zbq"
server_key: "f1e1a6acb5de0f9d5885149df99fb923"
server_key_iv: "8b037745e7563c06545bf1c5f1f9f4a9"
server_key_version: "encryptionkey@0"
I20260806 06:34:54.497929 1677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:34:54.533140 1677 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:34:54.535059 1677 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:34:54.535187 1677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:34:54.535374 1677 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:34:54.535643 1677 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:34:54.535703 1677 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:54.535740 1677 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:34:54.535765 1677 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:54.541074 1677 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.65:46149
I20260806 06:34:54.541149 1859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.65:46149 every 8 connection(s)
I20260806 06:34:54.541404 1677 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
I20260806 06:34:54.544922 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 1677
I20260806 06:34:54.545044 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/ts-0/wal/instance
I20260806 06:34:54.545240 7341 external_mini_cluster.cc:1428] Setting key dbcb8c869ff425b772af3eb7d3b59309
Aug 06 06:34:54 dist-test-slave-6zbq krb5kdc[1517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998094, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.65@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:34:54.549893 1644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:54.542041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:50707 (local address 127.7.43.126:43743)
0806 06:34:54.542088 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:54.542091 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:54.542277 (+ 186us) server_negotiation.cc:407] Connection header received
0806 06:34:54.542714 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:54.542715 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:54.542725 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:54.542770 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:34:54.543574 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:54.544046 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:54.545032 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:54.545291 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:54.545350 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:54.547383 (+ 2033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:34:54.547385 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:34:54.547385 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:34:54.547394 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:34:54.548679 (+ 1285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:34:54.549023 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:34:54.549024 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:34:54.549025 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:34:54.549083 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:34:54.549297 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:34:54.549298 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:34:54.549299 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:34:54.549414 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:34:54.549469 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:54.549707 (+ 238us) server_negotiation.cc:299] Negotiation successful
0806 06:34:54.549803 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 06:34:54.550119 1865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:54.542022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:43743 (local address 127.7.43.65:50707)
0806 06:34:54.542174 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:54.542205 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:54.542636 (+ 431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:54.542826 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:54.542829 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:54.543341 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:34:54.543510 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:54.543514 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:54.544104 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:54.544105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:54.544945 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:54.544948 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:54.545375 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:54.545376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:54.545500 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:54.545902 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:34:54.545909 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:34:54.547285 (+ 1376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:34:54.548744 (+ 1459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:34:54.548750 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:34:54.548752 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:34:54.548949 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:34:54.549137 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:34:54.549138 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:34:54.549138 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:34:54.549237 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:34:54.549466 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:34:54.549473 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:34:54.549595 (+ 122us) client_negotiation.cc:769] Sending connection context
0806 06:34:54.549678 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 06:34:54.549816 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":84,"threads_started":1}
I20260806 06:34:54.550580 1860 heartbeater.cc:347] Connected to a master server at 127.7.43.126:43743
I20260806 06:34:54.550660 1860 heartbeater.cc:464] Registering TS with master...
I20260806 06:34:54.550815 1860 heartbeater.cc:511] Master 127.7.43.126:43743 requested a full tablet report, sending...
I20260806 06:34:54.551282 1583 ts_manager.cc:194] Registered new tserver with Master: 04c42cac64f54ff98de03eef3df52b74 (127.7.43.65:46149)
I20260806 06:34:54.551860 1583 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.65@KRBTEST.COM'} at 127.7.43.65:50707
WARNING: no policy specified for kudu/127.7.43.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab.
Entry for principal kudu/127.7.43.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab.
I20260806 06:34:54.568244 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.66:0
--local_ip_for_outbound_sockets=127.7.43.66
--webserver_interface=127.7.43.66
--webserver_port=0
--tserver_master_addrs=127.7.43.126:43743
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab
--principal=kudu/127.7.43.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:37381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:34:54.646533 1868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:34:54.646709 1868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:34:54.646731 1868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:34:54.647780 1868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:34:54.647812 1868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:34:54.647836 1868 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.66
I20260806 06:34:54.649185 1868 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:37381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab
--principal=kudu/127.7.43.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.43.66
--webserver_port=0
--tserver_master_addrs=127.7.43.126:43743
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1868
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.66
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:34:54.649370 1868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:34:54.649564 1868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:34:54.654050 1868 server_base.cc:1034] running on GCE node
W20260806 06:34:54.654031 1876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:34:54.654155 1879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:34:54.654036 1875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:34:54.654563 1868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:34:54.654786 1868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:34:54.655918 1868 hybrid_clock.cc:648] HybridClock initialized: now 1785998094655908 us; error 26 us; skew 500 ppm
Aug 06 06:34:54 dist-test-slave-6zbq krb5kdc[1517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998094, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:34:54.658113 1868 init.cc:377] Logged in from keytab as kudu/127.7.43.66@KRBTEST.COM (short username kudu)
I20260806 06:34:54.658535 1868 webserver.cc:460] Webserver started at http://127.7.43.66:38423/ using document root <none> and password file <none>
I20260806 06:34:54.658720 1868 fs_manager.cc:359] Metadata directory not provided
I20260806 06:34:54.658766 1868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:34:54.658888 1868 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:34:54.659811 1868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/ts-1/data/instance:
uuid: "a87a594ec2624f70ad5a5216c1268500"
format_stamp: "Formatted at 2026-08-06 06:34:54 on dist-test-slave-6zbq"
server_key: "7a9ddf422bec00c6ab2322824224e62a"
server_key_iv: "cace27a1abf6a1276042d961656249ae"
server_key_version: "encryptionkey@0"
I20260806 06:34:54.660125 1868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/ts-1/wal/instance:
uuid: "a87a594ec2624f70ad5a5216c1268500"
format_stamp: "Formatted at 2026-08-06 06:34:54 on dist-test-slave-6zbq"
server_key: "7a9ddf422bec00c6ab2322824224e62a"
server_key_iv: "cace27a1abf6a1276042d961656249ae"
server_key_version: "encryptionkey@0"
I20260806 06:34:54.661307 1868 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:34:54.661947 1888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:54.662076 1868 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:34:54.662155 1868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/ts-1/wal
uuid: "a87a594ec2624f70ad5a5216c1268500"
format_stamp: "Formatted at 2026-08-06 06:34:54 on dist-test-slave-6zbq"
server_key: "7a9ddf422bec00c6ab2322824224e62a"
server_key_iv: "cace27a1abf6a1276042d961656249ae"
server_key_version: "encryptionkey@0"
I20260806 06:34:54.662220 1868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:34:54.672789 1868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:34:54.674716 1868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:34:54.674842 1868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:34:54.675026 1868 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:34:54.675290 1868 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:34:54.675323 1868 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:54.675350 1868 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:34:54.675367 1868 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:54.680578 1868 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.66:41143
I20260806 06:34:54.680631 2052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.66:41143 every 8 connection(s)
I20260806 06:34:54.680878 1868 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
I20260806 06:34:54.682127 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 1868
I20260806 06:34:54.682235 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/ts-1/wal/instance
I20260806 06:34:54.682471 7341 external_mini_cluster.cc:1428] Setting key 50b7f56801c62aec810908a8680ecc00
Aug 06 06:34:54 dist-test-slave-6zbq krb5kdc[1517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998094, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.66@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:34:54.689806 1644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:54.681478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:48595 (local address 127.7.43.126:43743)
0806 06:34:54.681535 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:54.681536 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:54.681703 (+ 167us) server_negotiation.cc:407] Connection header received
0806 06:34:54.682145 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:54.682147 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:54.682159 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:54.682209 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:34:54.683279 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:54.683783 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:54.684788 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:54.684986 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:54.685034 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:54.687283 (+ 2249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:34:54.687284 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:34:54.687285 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:34:54.687293 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:34:54.688546 (+ 1253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:34:54.688984 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:34:54.688985 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:34:54.688985 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:34:54.689027 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:34:54.689266 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:34:54.689266 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:34:54.689266 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:34:54.689357 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:34:54.689400 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:54.689679 (+ 279us) server_negotiation.cc:299] Negotiation successful
0806 06:34:54.689738 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 06:34:54.690037 2055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:54.681457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:43743 (local address 127.7.43.66:48595)
0806 06:34:54.681594 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:54.681625 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:54.682051 (+ 426us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:54.682278 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:54.682282 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:54.682949 (+ 667us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:34:54.683201 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:54.683207 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:54.683860 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:54.683861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:54.684694 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:54.684696 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:54.685053 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:54.685054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:54.685185 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:54.685608 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:34:54.685617 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:34:54.687196 (+ 1579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:34:54.688623 (+ 1427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:34:54.688630 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:34:54.688633 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:34:54.688917 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:34:54.689076 (+ 159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:34:54.689077 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:34:54.689077 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:34:54.689210 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:34:54.689408 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:34:54.689413 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:34:54.689564 (+ 151us) client_negotiation.cc:769] Sending connection context
0806 06:34:54.689679 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 06:34:54.689833 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":73,"threads_started":1}
I20260806 06:34:54.690526 2053 heartbeater.cc:347] Connected to a master server at 127.7.43.126:43743
I20260806 06:34:54.690610 2053 heartbeater.cc:464] Registering TS with master...
I20260806 06:34:54.690768 2053 heartbeater.cc:511] Master 127.7.43.126:43743 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.7.43.67@KRBTEST.COM; defaulting to no policy
I20260806 06:34:54.691124 1583 ts_manager.cc:194] Registered new tserver with Master: a87a594ec2624f70ad5a5216c1268500 (127.7.43.66:41143)
I20260806 06:34:54.691550 1583 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.66@KRBTEST.COM'} at 127.7.43.66:48595
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab.
Entry for principal kudu/127.7.43.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab.
I20260806 06:34:54.705677 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.67:0
--local_ip_for_outbound_sockets=127.7.43.67
--webserver_interface=127.7.43.67
--webserver_port=0
--tserver_master_addrs=127.7.43.126:43743
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab
--principal=kudu/127.7.43.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:37381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:34:54.782763 2059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:34:54.782940 2059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:34:54.782970 2059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:34:54.784034 2059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:34:54.784072 2059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:34:54.784106 2059 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.67
I20260806 06:34:54.785460 2059 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:37381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab
--principal=kudu/127.7.43.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.43.67
--webserver_port=0
--tserver_master_addrs=127.7.43.126:43743
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2059
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.67
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:34:54.785677 2059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:34:54.785866 2059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:34:54.790396 2072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:34:54.790405 2066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:34:54.790400 2067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:34:54.790555 2059 server_base.cc:1034] running on GCE node
I20260806 06:34:54.790784 2059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:34:54.790988 2059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:34:54.792129 2059 hybrid_clock.cc:648] HybridClock initialized: now 1785998094792118 us; error 29 us; skew 500 ppm
Aug 06 06:34:54 dist-test-slave-6zbq krb5kdc[1517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998094, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:34:54.794386 2059 init.cc:377] Logged in from keytab as kudu/127.7.43.67@KRBTEST.COM (short username kudu)
I20260806 06:34:54.794772 2059 webserver.cc:460] Webserver started at http://127.7.43.67:41151/ using document root <none> and password file <none>
I20260806 06:34:54.794931 2059 fs_manager.cc:359] Metadata directory not provided
I20260806 06:34:54.794967 2059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:34:54.795051 2059 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:34:54.795923 2059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/ts-2/data/instance:
uuid: "95b631bd63524bf0aad3979279eeb203"
format_stamp: "Formatted at 2026-08-06 06:34:54 on dist-test-slave-6zbq"
server_key: "881c7c3baff6391e357cfcb3c4d768c8"
server_key_iv: "d93a5484eb854a217a250f0c666eaed3"
server_key_version: "encryptionkey@0"
I20260806 06:34:54.796197 2059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/ts-2/wal/instance:
uuid: "95b631bd63524bf0aad3979279eeb203"
format_stamp: "Formatted at 2026-08-06 06:34:54 on dist-test-slave-6zbq"
server_key: "881c7c3baff6391e357cfcb3c4d768c8"
server_key_iv: "d93a5484eb854a217a250f0c666eaed3"
server_key_version: "encryptionkey@0"
I20260806 06:34:54.797272 2059 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:34:54.797940 2080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:54.798091 2059 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:54.798172 2059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/ts-2/wal
uuid: "95b631bd63524bf0aad3979279eeb203"
format_stamp: "Formatted at 2026-08-06 06:34:54 on dist-test-slave-6zbq"
server_key: "881c7c3baff6391e357cfcb3c4d768c8"
server_key_iv: "d93a5484eb854a217a250f0c666eaed3"
server_key_version: "encryptionkey@0"
I20260806 06:34:54.798239 2059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:34:54.805303 2059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:34:54.806972 2059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:34:54.807092 2059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:34:54.807276 2059 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:34:54.807526 2059 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:34:54.807559 2059 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:54.807583 2059 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:34:54.807607 2059 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:54.812960 2059 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.67:37329
I20260806 06:34:54.812994 2247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.67:37329 every 8 connection(s)
I20260806 06:34:54.813251 2059 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
Aug 06 06:34:54 dist-test-slave-6zbq krb5kdc[1517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998094, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.67@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:34:54.819620 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 2059
I20260806 06:34:54.819736 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/ts-2/wal/instance
I20260806 06:34:54.819913 7341 external_mini_cluster.cc:1428] Setting key a236561185dc13341f56d699eefd42e2
I20260806 06:34:54.821437 1644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:54.813842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:39187 (local address 127.7.43.126:43743)
0806 06:34:54.813901 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:54.813903 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:54.814070 (+ 167us) server_negotiation.cc:407] Connection header received
0806 06:34:54.814515 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:54.814516 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:54.814526 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:54.814565 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:34:54.815412 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:54.815886 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:54.816793 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:54.816970 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:54.817025 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:54.819086 (+ 2061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:34:54.819088 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:34:54.819088 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:34:54.819094 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:34:54.820309 (+ 1215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:34:54.820711 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:34:54.820712 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:34:54.820713 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:34:54.820777 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:34:54.820982 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:34:54.820983 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:34:54.820983 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:34:54.821080 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:34:54.821139 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:54.821320 (+ 181us) server_negotiation.cc:299] Negotiation successful
0806 06:34:54.821377 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 06:34:54.821569 2251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:54.813817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:43743 (local address 127.7.43.67:39187)
0806 06:34:54.813972 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:54.814001 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:54.814429 (+ 428us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:54.814620 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:54.814623 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:54.815166 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:34:54.815357 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:54.815361 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:54.815949 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:54.815949 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:54.816732 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:54.816735 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:54.817046 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:54.817047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:54.817180 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:54.817657 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:34:54.817666 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:34:54.818999 (+ 1333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:34:54.820365 (+ 1366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:34:54.820372 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:34:54.820375 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:34:54.820606 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:34:54.820840 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:34:54.820841 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:34:54.820841 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:34:54.820941 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:34:54.821132 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:34:54.821134 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:34:54.821236 (+ 102us) client_negotiation.cc:769] Sending connection context
0806 06:34:54.821305 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 06:34:54.821410 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":90,"threads_started":1}
I20260806 06:34:54.821966 2248 heartbeater.cc:347] Connected to a master server at 127.7.43.126:43743
I20260806 06:34:54.822044 2248 heartbeater.cc:464] Registering TS with master...
I20260806 06:34:54.822193 2248 heartbeater.cc:511] Master 127.7.43.126:43743 requested a full tablet report, sending...
I20260806 06:34:54.822531 1583 ts_manager.cc:194] Registered new tserver with Master: 95b631bd63524bf0aad3979279eeb203 (127.7.43.67:37329)
I20260806 06:34:54.822880 1583 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.67@KRBTEST.COM'} at 127.7.43.67:39187
I20260806 06:34:54.831426 7341 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:34:54.837548 1644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:54.832603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44168 (local address 127.7.43.126:43743)
0806 06:34:54.832657 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:54.832659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:54.832752 (+ 93us) server_negotiation.cc:407] Connection header received
0806 06:34:54.832789 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:54.832790 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:54.832797 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:54.832838 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:34:54.833307 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:54.833729 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:54.834377 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:54.834548 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:54.834590 (+ 42us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:54.835593 (+ 1003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:34:54.835594 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:34:54.835594 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:34:54.835599 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:34:54.836683 (+ 1084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:34:54.837002 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:34:54.837003 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:34:54.837003 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:34:54.837046 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:34:54.837175 (+ 129us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:34:54.837176 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:34:54.837176 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:34:54.837284 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:34:54.837351 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:54.837435 (+ 84us) server_negotiation.cc:299] Negotiation successful
0806 06:34:54.837476 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 06:34:54.838883 1583 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44168:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:34:54.840520 1583 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:34:54.846127 2263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:54.842654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.66:41143 (local address 127.0.0.1:37990)
0806 06:34:54.842734 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:54.842743 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:54.842789 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:54.842980 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:54.842982 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:54.842983 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:34:54.843032 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:54.843033 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:54.844023 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:54.844024 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:54.845005 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:54.845008 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:54.845855 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:54.845856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:54.845933 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:54.845945 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 06:34:54.846011 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 06:34:54.846055 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69,"mutex_wait_us":1,"thread_start_us":50,"threads_started":1}
I20260806 06:34:54.846155 2261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:54.842483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37990 (local address 127.7.43.66:41143)
0806 06:34:54.842833 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:54.842837 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:54.842847 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:34:54.842885 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:54.842887 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:54.842900 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:54.842962 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:34:54.843101 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:54.843935 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:54.845090 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:54.845786 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:54.845849 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:54.845905 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:54.846030 (+ 125us) server_negotiation.cc:299] Negotiation successful
0806 06:34:54.846075 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":93,"threads_started":1}
I20260806 06:34:54.846657 2266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:54.842717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43648 (local address 127.7.43.67:37329)
0806 06:34:54.843313 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:54.843317 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:54.843325 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:34:54.843354 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:54.843356 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:54.843369 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:54.843565 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:34:54.843616 (+ 51us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:54.844596 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:54.845700 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:54.846376 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:54.846433 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:54.846482 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:54.846557 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 06:34:54.846591 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":70,"threads_started":1}
I20260806 06:34:54.846647 2262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:54.842581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.67:37329 (local address 127.0.0.1:43648)
0806 06:34:54.842655 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:54.842664 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:54.842704 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:54.843463 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:54.843464 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:54.843466 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:34:54.843499 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:54.843501 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:54.844680 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:54.844682 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:54.845632 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:54.845634 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:54.846433 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:54.846434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:54.846491 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:54.846494 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:34:54.846544 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 06:34:54.846580 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61,"thread_start_us":40,"threads_started":1}
I20260806 06:34:54.846951 2260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:54.842480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33938 (local address 127.7.43.65:46149)
0806 06:34:54.843072 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:54.843075 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:54.843082 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:34:54.843111 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:54.843113 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:54.843124 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:54.843182 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:34:54.843469 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:54.844553 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:54.845829 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:54.846522 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:54.846567 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:54.846618 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:54.846832 (+ 214us) server_negotiation.cc:299] Negotiation successful
0806 06:34:54.846872 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":89,"threads_started":1}
I20260806 06:34:54.847039 1961 tablet_service.cc:1467] Processing CreateTablet for tablet 4c8bab8ff3574187bce89c872f54dc57 (DEFAULT_TABLE table=test-table [id=048cb9ab01c84862bdba1bff7819f115]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:34:54.847159 2259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:54.842405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:46149 (local address 127.0.0.1:33938)
0806 06:34:54.842501 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:54.842538 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:54.842601 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:54.843199 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:54.843200 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:54.843203 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:34:54.843327 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:54.843330 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:54.844668 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:54.844669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:54.845727 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:54.845729 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:54.846675 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:54.846675 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:54.846745 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:54.846748 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:34:54.847057 (+ 309us) client_negotiation.cc:240] Negotiation successful
0806 06:34:54.847098 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72,"mutex_wait_us":37,"thread_start_us":51,"threads_started":1}
I20260806 06:34:54.847393 1961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c8bab8ff3574187bce89c872f54dc57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:34:54.849741 2267 tablet_bootstrap.cc:492] T 4c8bab8ff3574187bce89c872f54dc57 P a87a594ec2624f70ad5a5216c1268500: Bootstrap starting.
I20260806 06:34:54.850040 2149 tablet_service.cc:1467] Processing CreateTablet for tablet 4c8bab8ff3574187bce89c872f54dc57 (DEFAULT_TABLE table=test-table [id=048cb9ab01c84862bdba1bff7819f115]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:34:54.850284 2149 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c8bab8ff3574187bce89c872f54dc57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:34:54.850395 2267 tablet_bootstrap.cc:654] T 4c8bab8ff3574187bce89c872f54dc57 P a87a594ec2624f70ad5a5216c1268500: Neither blocks nor log segments found. Creating new log.
I20260806 06:34:54.850889 2267 log.cc:824] T 4c8bab8ff3574187bce89c872f54dc57 P a87a594ec2624f70ad5a5216c1268500: Log is configured to *not* fsync() on all Append() calls
I20260806 06:34:54.851502 2267 tablet_bootstrap.cc:492] T 4c8bab8ff3574187bce89c872f54dc57 P a87a594ec2624f70ad5a5216c1268500: No bootstrap required, opened a new log
I20260806 06:34:54.851542 2267 ts_tablet_manager.cc:1397] T 4c8bab8ff3574187bce89c872f54dc57 P a87a594ec2624f70ad5a5216c1268500: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:34:54.852732 2270 tablet_bootstrap.cc:492] T 4c8bab8ff3574187bce89c872f54dc57 P 95b631bd63524bf0aad3979279eeb203: Bootstrap starting.
I20260806 06:34:54.852874 2267 raft_consensus.cc:348] T 4c8bab8ff3574187bce89c872f54dc57 P a87a594ec2624f70ad5a5216c1268500 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04c42cac64f54ff98de03eef3df52b74" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 46149 } } peers { permanent_uuid: "a87a594ec2624f70ad5a5216c1268500" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 41143 } } peers { permanent_uuid: "95b631bd63524bf0aad3979279eeb203" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 37329 } }
I20260806 06:34:54.853012 2267 raft_consensus.cc:374] T 4c8bab8ff3574187bce89c872f54dc57 P a87a594ec2624f70ad5a5216c1268500 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:34:54.853032 2267 raft_consensus.cc:729] T 4c8bab8ff3574187bce89c872f54dc57 P a87a594ec2624f70ad5a5216c1268500 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a87a594ec2624f70ad5a5216c1268500, State: Initialized, Role: FOLLOWER
I20260806 06:34:54.853117 2267 consensus_queue.cc:260] T 4c8bab8ff3574187bce89c872f54dc57 P a87a594ec2624f70ad5a5216c1268500 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04c42cac64f54ff98de03eef3df52b74" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 46149 } } peers { permanent_uuid: "a87a594ec2624f70ad5a5216c1268500" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 41143 } } peers { permanent_uuid: "95b631bd63524bf0aad3979279eeb203" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 37329 } }
I20260806 06:34:54.853114 1770 tablet_service.cc:1467] Processing CreateTablet for tablet 4c8bab8ff3574187bce89c872f54dc57 (DEFAULT_TABLE table=test-table [id=048cb9ab01c84862bdba1bff7819f115]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:34:54.853336 2270 tablet_bootstrap.cc:654] T 4c8bab8ff3574187bce89c872f54dc57 P 95b631bd63524bf0aad3979279eeb203: Neither blocks nor log segments found. Creating new log.
I20260806 06:34:54.853436 1770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c8bab8ff3574187bce89c872f54dc57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:34:54.853471 2267 ts_tablet_manager.cc:1428] T 4c8bab8ff3574187bce89c872f54dc57 P a87a594ec2624f70ad5a5216c1268500: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:34:54.853854 2270 log.cc:824] T 4c8bab8ff3574187bce89c872f54dc57 P 95b631bd63524bf0aad3979279eeb203: Log is configured to *not* fsync() on all Append() calls
I20260806 06:34:54.854437 2053 heartbeater.cc:503] Master 127.7.43.126:43743 was elected leader, sending a full tablet report...
I20260806 06:34:54.855471 2270 tablet_bootstrap.cc:492] T 4c8bab8ff3574187bce89c872f54dc57 P 95b631bd63524bf0aad3979279eeb203: No bootstrap required, opened a new log
I20260806 06:34:54.855533 2270 ts_tablet_manager.cc:1397] T 4c8bab8ff3574187bce89c872f54dc57 P 95b631bd63524bf0aad3979279eeb203: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 06:34:54.856113 2274 tablet_bootstrap.cc:492] T 4c8bab8ff3574187bce89c872f54dc57 P 04c42cac64f54ff98de03eef3df52b74: Bootstrap starting.
I20260806 06:34:54.856648 2274 tablet_bootstrap.cc:654] T 4c8bab8ff3574187bce89c872f54dc57 P 04c42cac64f54ff98de03eef3df52b74: Neither blocks nor log segments found. Creating new log.
I20260806 06:34:54.856866 2270 raft_consensus.cc:348] T 4c8bab8ff3574187bce89c872f54dc57 P 95b631bd63524bf0aad3979279eeb203 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04c42cac64f54ff98de03eef3df52b74" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 46149 } } peers { permanent_uuid: "a87a594ec2624f70ad5a5216c1268500" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 41143 } } peers { permanent_uuid: "95b631bd63524bf0aad3979279eeb203" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 37329 } }
I20260806 06:34:54.856988 2270 raft_consensus.cc:374] T 4c8bab8ff3574187bce89c872f54dc57 P 95b631bd63524bf0aad3979279eeb203 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:34:54.857019 2270 raft_consensus.cc:729] T 4c8bab8ff3574187bce89c872f54dc57 P 95b631bd63524bf0aad3979279eeb203 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95b631bd63524bf0aad3979279eeb203, State: Initialized, Role: FOLLOWER
I20260806 06:34:54.857057 2274 log.cc:824] T 4c8bab8ff3574187bce89c872f54dc57 P 04c42cac64f54ff98de03eef3df52b74: Log is configured to *not* fsync() on all Append() calls
I20260806 06:34:54.857121 2270 consensus_queue.cc:260] T 4c8bab8ff3574187bce89c872f54dc57 P 95b631bd63524bf0aad3979279eeb203 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04c42cac64f54ff98de03eef3df52b74" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 46149 } } peers { permanent_uuid: "a87a594ec2624f70ad5a5216c1268500" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 41143 } } peers { permanent_uuid: "95b631bd63524bf0aad3979279eeb203" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 37329 } }
I20260806 06:34:54.857378 2270 ts_tablet_manager.cc:1428] T 4c8bab8ff3574187bce89c872f54dc57 P 95b631bd63524bf0aad3979279eeb203: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:34:54.857386 2248 heartbeater.cc:503] Master 127.7.43.126:43743 was elected leader, sending a full tablet report...
I20260806 06:34:54.857812 2274 tablet_bootstrap.cc:492] T 4c8bab8ff3574187bce89c872f54dc57 P 04c42cac64f54ff98de03eef3df52b74: No bootstrap required, opened a new log
I20260806 06:34:54.857869 2274 ts_tablet_manager.cc:1397] T 4c8bab8ff3574187bce89c872f54dc57 P 04c42cac64f54ff98de03eef3df52b74: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:34:54.859411 2274 raft_consensus.cc:348] T 4c8bab8ff3574187bce89c872f54dc57 P 04c42cac64f54ff98de03eef3df52b74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04c42cac64f54ff98de03eef3df52b74" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 46149 } } peers { permanent_uuid: "a87a594ec2624f70ad5a5216c1268500" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 41143 } } peers { permanent_uuid: "95b631bd63524bf0aad3979279eeb203" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 37329 } }
I20260806 06:34:54.859541 2274 raft_consensus.cc:374] T 4c8bab8ff3574187bce89c872f54dc57 P 04c42cac64f54ff98de03eef3df52b74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:34:54.859561 2274 raft_consensus.cc:729] T 4c8bab8ff3574187bce89c872f54dc57 P 04c42cac64f54ff98de03eef3df52b74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04c42cac64f54ff98de03eef3df52b74, State: Initialized, Role: FOLLOWER
I20260806 06:34:54.859665 2274 consensus_queue.cc:260] T 4c8bab8ff3574187bce89c872f54dc57 P 04c42cac64f54ff98de03eef3df52b74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04c42cac64f54ff98de03eef3df52b74" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 46149 } } peers { permanent_uuid: "a87a594ec2624f70ad5a5216c1268500" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 41143 } } peers { permanent_uuid: "95b631bd63524bf0aad3979279eeb203" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 37329 } }
I20260806 06:34:54.859895 2274 ts_tablet_manager.cc:1428] T 4c8bab8ff3574187bce89c872f54dc57 P 04c42cac64f54ff98de03eef3df52b74: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:34:54.859925 1860 heartbeater.cc:503] Master 127.7.43.126:43743 was elected leader, sending a full tablet report...
I20260806 06:34:54.863896 2279 raft_consensus.cc:482] T 4c8bab8ff3574187bce89c872f54dc57 P 04c42cac64f54ff98de03eef3df52b74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:34:54.863958 2279 raft_consensus.cc:504] T 4c8bab8ff3574187bce89c872f54dc57 P 04c42cac64f54ff98de03eef3df52b74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04c42cac64f54ff98de03eef3df52b74" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 46149 } } peers { permanent_uuid: "a87a594ec2624f70ad5a5216c1268500" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 41143 } } peers { permanent_uuid: "95b631bd63524bf0aad3979279eeb203" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 37329 } }
I20260806 06:34:54.864174 2279 leader_election.cc:288] T 4c8bab8ff3574187bce89c872f54dc57 P 04c42cac64f54ff98de03eef3df52b74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a87a594ec2624f70ad5a5216c1268500 (127.7.43.66:41143), 95b631bd63524bf0aad3979279eeb203 (127.7.43.67:37329)
I20260806 06:34:54.866770 2261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:54.864436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:53863 (local address 127.7.43.66:41143)
0806 06:34:54.864472 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:54.864474 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:54.864481 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:34:54.864498 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:54.864498 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:54.864506 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:54.864556 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:34:54.864676 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:54.865126 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:54.866074 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:54.866469 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:54.866522 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:54.866532 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:54.866674 (+ 142us) server_negotiation.cc:299] Negotiation successful
0806 06:34:54.866702 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 06:34:54.866773 2266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:54.864379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:50695 (local address 127.7.43.67:37329)
0806 06:34:54.864428 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:54.864430 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:54.864533 (+ 103us) server_negotiation.cc:407] Connection header received
0806 06:34:54.864582 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:54.864582 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:54.864592 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:54.864634 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:34:54.864754 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:54.865168 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:54.866077 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:54.866481 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:54.866533 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:54.866542 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:54.866686 (+ 144us) server_negotiation.cc:299] Negotiation successful
0806 06:34:54.866716 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 06:34:54.866784 1865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:54.864302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.66:41143 (local address 127.7.43.65:53863)
0806 06:34:54.864347 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:54.864355 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:54.864388 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:54.864571 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:54.864572 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:54.864573 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:34:54.864623 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:54.864625 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:54.865194 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:54.865195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:54.866008 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:54.866010 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:54.866542 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:54.866543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:54.866619 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:54.866622 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:34:54.866664 (+ 42us) client_negotiation.cc:240] Negotiation successful
0806 06:34:54.866702 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 06:34:54.866802 2281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:54.864313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.67:37329 (local address 127.7.43.65:50695)
0806 06:34:54.864476 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:54.864484 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:54.864530 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:54.864636 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:54.864637 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:54.864638 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:34:54.864704 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:54.864705 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:54.865238 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:54.865239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:54.866001 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:54.866003 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:54.866556 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:54.866557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:54.866629 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:54.866632 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:34:54.866681 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 06:34:54.866717 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":82,"threads_started":1}
I20260806 06:34:54.867102 1993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c8bab8ff3574187bce89c872f54dc57" candidate_uuid: "04c42cac64f54ff98de03eef3df52b74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a87a594ec2624f70ad5a5216c1268500" is_pre_election: true
I20260806 06:34:54.867233 1993 raft_consensus.cc:2454] T 4c8bab8ff3574187bce89c872f54dc57 P a87a594ec2624f70ad5a5216c1268500 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04c42cac64f54ff98de03eef3df52b74 in term 0.
I20260806 06:34:54.867273 2176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c8bab8ff3574187bce89c872f54dc57" candidate_uuid: "04c42cac64f54ff98de03eef3df52b74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95b631bd63524bf0aad3979279eeb203" is_pre_election: true
I20260806 06:34:54.867403 2176 raft_consensus.cc:2454] T 4c8bab8ff3574187bce89c872f54dc57 P 95b631bd63524bf0aad3979279eeb203 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04c42cac64f54ff98de03eef3df52b74 in term 0.
I20260806 06:34:54.867437 1699 leader_election.cc:302] T 4c8bab8ff3574187bce89c872f54dc57 P 04c42cac64f54ff98de03eef3df52b74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04c42cac64f54ff98de03eef3df52b74, a87a594ec2624f70ad5a5216c1268500; no voters:
I20260806 06:34:54.867566 2279 raft_consensus.cc:2781] T 4c8bab8ff3574187bce89c872f54dc57 P 04c42cac64f54ff98de03eef3df52b74 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:34:54.867612 2279 raft_consensus.cc:482] T 4c8bab8ff3574187bce89c872f54dc57 P 04c42cac64f54ff98de03eef3df52b74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:34:54.867630 2279 raft_consensus.cc:3037] T 4c8bab8ff3574187bce89c872f54dc57 P 04c42cac64f54ff98de03eef3df52b74 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:34:54.868124 2279 raft_consensus.cc:504] T 4c8bab8ff3574187bce89c872f54dc57 P 04c42cac64f54ff98de03eef3df52b74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04c42cac64f54ff98de03eef3df52b74" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 46149 } } peers { permanent_uuid: "a87a594ec2624f70ad5a5216c1268500" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 41143 } } peers { permanent_uuid: "95b631bd63524bf0aad3979279eeb203" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 37329 } }
I20260806 06:34:54.868233 2279 leader_election.cc:288] T 4c8bab8ff3574187bce89c872f54dc57 P 04c42cac64f54ff98de03eef3df52b74 [CANDIDATE]: Term 1 election: Requested vote from peers a87a594ec2624f70ad5a5216c1268500 (127.7.43.66:41143), 95b631bd63524bf0aad3979279eeb203 (127.7.43.67:37329)
I20260806 06:34:54.868367 2176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c8bab8ff3574187bce89c872f54dc57" candidate_uuid: "04c42cac64f54ff98de03eef3df52b74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95b631bd63524bf0aad3979279eeb203"
I20260806 06:34:54.868429 2176 raft_consensus.cc:3037] T 4c8bab8ff3574187bce89c872f54dc57 P 95b631bd63524bf0aad3979279eeb203 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:34:54.868428 1993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c8bab8ff3574187bce89c872f54dc57" candidate_uuid: "04c42cac64f54ff98de03eef3df52b74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a87a594ec2624f70ad5a5216c1268500"
I20260806 06:34:54.868489 1993 raft_consensus.cc:3037] T 4c8bab8ff3574187bce89c872f54dc57 P a87a594ec2624f70ad5a5216c1268500 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:34:54.869020 2176 raft_consensus.cc:2454] T 4c8bab8ff3574187bce89c872f54dc57 P 95b631bd63524bf0aad3979279eeb203 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04c42cac64f54ff98de03eef3df52b74 in term 1.
I20260806 06:34:54.869112 1993 raft_consensus.cc:2454] T 4c8bab8ff3574187bce89c872f54dc57 P a87a594ec2624f70ad5a5216c1268500 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04c42cac64f54ff98de03eef3df52b74 in term 1.
I20260806 06:34:54.869187 1698 leader_election.cc:302] T 4c8bab8ff3574187bce89c872f54dc57 P 04c42cac64f54ff98de03eef3df52b74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04c42cac64f54ff98de03eef3df52b74, 95b631bd63524bf0aad3979279eeb203; no voters:
I20260806 06:34:54.869270 2279 raft_consensus.cc:2781] T 4c8bab8ff3574187bce89c872f54dc57 P 04c42cac64f54ff98de03eef3df52b74 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:34:54.869396 2279 raft_consensus.cc:686] T 4c8bab8ff3574187bce89c872f54dc57 P 04c42cac64f54ff98de03eef3df52b74 [term 1 LEADER]: Becoming Leader. State: Replica: 04c42cac64f54ff98de03eef3df52b74, State: Running, Role: LEADER
I20260806 06:34:54.869524 2279 consensus_queue.cc:237] T 4c8bab8ff3574187bce89c872f54dc57 P 04c42cac64f54ff98de03eef3df52b74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04c42cac64f54ff98de03eef3df52b74" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 46149 } } peers { permanent_uuid: "a87a594ec2624f70ad5a5216c1268500" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 41143 } } peers { permanent_uuid: "95b631bd63524bf0aad3979279eeb203" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 37329 } }
I20260806 06:34:54.870260 1582 catalog_manager.cc:5495] T 4c8bab8ff3574187bce89c872f54dc57 P 04c42cac64f54ff98de03eef3df52b74 reported cstate change: term changed from 0 to 1, leader changed from <none> to 04c42cac64f54ff98de03eef3df52b74 (127.7.43.65). New cstate: current_term: 1 leader_uuid: "04c42cac64f54ff98de03eef3df52b74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04c42cac64f54ff98de03eef3df52b74" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 46149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a87a594ec2624f70ad5a5216c1268500" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 41143 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95b631bd63524bf0aad3979279eeb203" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 37329 } health_report { overall_health: UNKNOWN } } }
I20260806 06:34:54.881125 2260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:54.879021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33952 (local address 127.7.43.65:46149)
0806 06:34:54.879074 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:54.879076 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:54.879084 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:34:54.879130 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:54.879131 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:54.879148 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:54.879203 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:34:54.879315 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:54.879745 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:54.880424 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:54.880603 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:54.880654 (+ 51us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:54.880820 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:34:54.880898 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:34:54.880976 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:54.881025 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 06:34:54.881031 (+ 6us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 06:34:54.882498 1993 raft_consensus.cc:1261] T 4c8bab8ff3574187bce89c872f54dc57 P a87a594ec2624f70ad5a5216c1268500 [term 1 FOLLOWER]: Refusing update from remote peer 04c42cac64f54ff98de03eef3df52b74: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:34:54.882505 2176 raft_consensus.cc:1261] T 4c8bab8ff3574187bce89c872f54dc57 P 95b631bd63524bf0aad3979279eeb203 [term 1 FOLLOWER]: Refusing update from remote peer 04c42cac64f54ff98de03eef3df52b74: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:34:54.882807 2279 consensus_queue.cc:1035] T 4c8bab8ff3574187bce89c872f54dc57 P 04c42cac64f54ff98de03eef3df52b74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a87a594ec2624f70ad5a5216c1268500" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 41143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:34:54.882938 2282 consensus_queue.cc:1035] T 4c8bab8ff3574187bce89c872f54dc57 P 04c42cac64f54ff98de03eef3df52b74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95b631bd63524bf0aad3979279eeb203" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 37329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:34:54.885843 2288 mvcc.cc:205] Tried to move back new op lower bound from 7315448196636536832 to 7315448196586237952. Current Snapshot: MvccSnapshot[applied={T|T < 7315448196636536832}]
I20260806 06:34:54.885910 2287 mvcc.cc:205] Tried to move back new op lower bound from 7315448196636536832 to 7315448196586237952. Current Snapshot: MvccSnapshot[applied={T|T < 7315448196636536832 or (T in {7315448196636536832})}]
I20260806 06:34:54.887363 1583 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44168:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:34:54.887452 1583 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44168:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:34:54.888077 1583 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ad0fabc5ddf645319a6fcb30fbe2f6e3: Sending DeleteTablet for 3 replicas of tablet 4c8bab8ff3574187bce89c872f54dc57
I20260806 06:34:54.888372 1961 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c8bab8ff3574187bce89c872f54dc57 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:34:54 UTC) from {username='kudu', principal='kudu/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:37990
I20260806 06:34:54.888420 2149 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c8bab8ff3574187bce89c872f54dc57 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:34:54 UTC) from {username='kudu', principal='kudu/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:43648
I20260806 06:34:54.888581 2300 tablet_replica.cc:330] T 4c8bab8ff3574187bce89c872f54dc57 P a87a594ec2624f70ad5a5216c1268500: stopping tablet replica
I20260806 06:34:54.888675 2300 raft_consensus.cc:2229] T 4c8bab8ff3574187bce89c872f54dc57 P a87a594ec2624f70ad5a5216c1268500 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:34:54.888710 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 1677
I20260806 06:34:54.896916 2301 tablet_replica.cc:330] T 4c8bab8ff3574187bce89c872f54dc57 P 95b631bd63524bf0aad3979279eeb203: stopping tablet replica
I20260806 06:34:54.897045 2301 raft_consensus.cc:2229] T 4c8bab8ff3574187bce89c872f54dc57 P 95b631bd63524bf0aad3979279eeb203 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:34:54.897169 2301 raft_consensus.cc:2258] T 4c8bab8ff3574187bce89c872f54dc57 P 95b631bd63524bf0aad3979279eeb203 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:34:54.897840 2301 ts_tablet_manager.cc:1905] T 4c8bab8ff3574187bce89c872f54dc57 P 95b631bd63524bf0aad3979279eeb203: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:34:54.899626 2301 ts_tablet_manager.cc:1918] T 4c8bab8ff3574187bce89c872f54dc57 P 95b631bd63524bf0aad3979279eeb203: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:34:54.899724 2301 log.cc:1196] T 4c8bab8ff3574187bce89c872f54dc57 P 95b631bd63524bf0aad3979279eeb203: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/ts-2/wal/wals/4c8bab8ff3574187bce89c872f54dc57
I20260806 06:34:54.899858 2300 raft_consensus.cc:2258] T 4c8bab8ff3574187bce89c872f54dc57 P a87a594ec2624f70ad5a5216c1268500 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:34:54.899997 2301 ts_tablet_manager.cc:1939] T 4c8bab8ff3574187bce89c872f54dc57 P 95b631bd63524bf0aad3979279eeb203: Deleting consensus metadata
I20260806 06:34:54.900239 2300 ts_tablet_manager.cc:1905] T 4c8bab8ff3574187bce89c872f54dc57 P a87a594ec2624f70ad5a5216c1268500: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:34:54.900416 1566 catalog_manager.cc:4841] TS 95b631bd63524bf0aad3979279eeb203 (127.7.43.67:37329): tablet 4c8bab8ff3574187bce89c872f54dc57 (table test-table [id=048cb9ab01c84862bdba1bff7819f115]) successfully deleted
I20260806 06:34:54.901414 2300 ts_tablet_manager.cc:1918] T 4c8bab8ff3574187bce89c872f54dc57 P a87a594ec2624f70ad5a5216c1268500: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:34:54.901472 2300 log.cc:1196] T 4c8bab8ff3574187bce89c872f54dc57 P a87a594ec2624f70ad5a5216c1268500: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785997861828271-7341-0/minicluster-data/ts-1/wal/wals/4c8bab8ff3574187bce89c872f54dc57
I20260806 06:34:54.901690 2300 ts_tablet_manager.cc:1939] T 4c8bab8ff3574187bce89c872f54dc57 P a87a594ec2624f70ad5a5216c1268500: Deleting consensus metadata
I20260806 06:34:54.902035 1568 catalog_manager.cc:4841] TS a87a594ec2624f70ad5a5216c1268500 (127.7.43.66:41143): tablet 4c8bab8ff3574187bce89c872f54dc57 (table test-table [id=048cb9ab01c84862bdba1bff7819f115]) successfully deleted
W20260806 06:34:54.903766 1566 connection.cc:535] client connection to 127.7.43.65:46149 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 06:34:54.903849 1566 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 06:34:54.904072 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 1868
I20260806 06:34:54.904234 2259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:54.904115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:46149 (local address 127.0.0.1:33962)
0806 06:34:54.904166 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:54.904207 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.65:46149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 06:34:54.907660 1566 catalog_manager.cc:4584] TS 04c42cac64f54ff98de03eef3df52b74 (127.7.43.65:46149): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4c8bab8ff3574187bce89c872f54dc57: Network error: Client connection negotiation failed: client connection to 127.7.43.65:46149: connect: Connection refused (error 111)
I20260806 06:34:54.910665 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 2059
I20260806 06:34:54.921667 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 1538
2026-08-06T06:34:54Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (1927 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-06T06:34:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:34:54Z Disabled control of system clock
I20260806 06:34:54.951900 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:40389
--webserver_interface=127.7.43.126
--webserver_port=0
--builtin_ntp_servers=127.7.43.84:44207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.43.126:40389
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 06:34:55.029325 2310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:34:55.029510 2310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:34:55.029527 2310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:34:55.030594 2310 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:34:55.030632 2310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:34:55.030642 2310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:34:55.030653 2310 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:34:55.030663 2310 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:34:55.031913 2310 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:44207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.43.126:40389
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:40389
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.43.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2310
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:34:55.032080 2310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:34:55.032285 2310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:34:55.036482 2318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:34:55.036624 2310 server_base.cc:1034] running on GCE node
W20260806 06:34:55.036496 2319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:34:55.036568 2324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:34:55.036918 2310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:34:55.037181 2310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:34:55.038312 2310 hybrid_clock.cc:648] HybridClock initialized: now 1785998095038298 us; error 31 us; skew 500 ppm
I20260806 06:34:55.039434 2310 webserver.cc:460] Webserver started at http://127.7.43.126:34205/ using document root <none> and password file <none>
I20260806 06:34:55.039645 2310 fs_manager.cc:359] Metadata directory not provided
I20260806 06:34:55.039695 2310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:34:55.039817 2310 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:34:55.040781 2310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/master-0/data/instance:
uuid: "47b57233eaa5474784ee83811d21319d"
format_stamp: "Formatted at 2026-08-06 06:34:55 on dist-test-slave-6zbq"
server_key: "a47bb3b2d9ef15c02d2ac3ae9d9e1589"
server_key_iv: "c2462fe51457fb757ae61449e75be768"
server_key_version: "encryptionkey@0"
I20260806 06:34:55.041131 2310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/master-0/wal/instance:
uuid: "47b57233eaa5474784ee83811d21319d"
format_stamp: "Formatted at 2026-08-06 06:34:55 on dist-test-slave-6zbq"
server_key: "a47bb3b2d9ef15c02d2ac3ae9d9e1589"
server_key_iv: "c2462fe51457fb757ae61449e75be768"
server_key_version: "encryptionkey@0"
I20260806 06:34:55.042300 2310 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:34:55.042986 2333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:55.043126 2310 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:34:55.043216 2310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "47b57233eaa5474784ee83811d21319d"
format_stamp: "Formatted at 2026-08-06 06:34:55 on dist-test-slave-6zbq"
server_key: "a47bb3b2d9ef15c02d2ac3ae9d9e1589"
server_key_iv: "c2462fe51457fb757ae61449e75be768"
server_key_version: "encryptionkey@0"
I20260806 06:34:55.043280 2310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:34:55.064630 2310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:34:55.064857 2310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:34:55.064967 2310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:34:55.068418 2310 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.126:40389
I20260806 06:34:55.068449 2407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.126:40389 every 8 connection(s)
I20260806 06:34:55.068789 2310 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:34:55.069720 2411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:34:55.071686 2411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47b57233eaa5474784ee83811d21319d: Bootstrap starting.
I20260806 06:34:55.072175 2411 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 47b57233eaa5474784ee83811d21319d: Neither blocks nor log segments found. Creating new log.
I20260806 06:34:55.072395 2411 log.cc:824] T 00000000000000000000000000000000 P 47b57233eaa5474784ee83811d21319d: Log is configured to *not* fsync() on all Append() calls
I20260806 06:34:55.072956 2411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47b57233eaa5474784ee83811d21319d: No bootstrap required, opened a new log
I20260806 06:34:55.074460 2411 raft_consensus.cc:348] T 00000000000000000000000000000000 P 47b57233eaa5474784ee83811d21319d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47b57233eaa5474784ee83811d21319d" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 40389 } }
I20260806 06:34:55.074581 2411 raft_consensus.cc:374] T 00000000000000000000000000000000 P 47b57233eaa5474784ee83811d21319d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:34:55.074601 2411 raft_consensus.cc:729] T 00000000000000000000000000000000 P 47b57233eaa5474784ee83811d21319d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47b57233eaa5474784ee83811d21319d, State: Initialized, Role: FOLLOWER
I20260806 06:34:55.074703 2411 consensus_queue.cc:260] T 00000000000000000000000000000000 P 47b57233eaa5474784ee83811d21319d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47b57233eaa5474784ee83811d21319d" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 40389 } }
I20260806 06:34:55.074784 2411 raft_consensus.cc:388] T 00000000000000000000000000000000 P 47b57233eaa5474784ee83811d21319d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:34:55.074811 2411 raft_consensus.cc:482] T 00000000000000000000000000000000 P 47b57233eaa5474784ee83811d21319d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:34:55.074842 2411 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 47b57233eaa5474784ee83811d21319d [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:34:55.075268 2411 raft_consensus.cc:504] T 00000000000000000000000000000000 P 47b57233eaa5474784ee83811d21319d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47b57233eaa5474784ee83811d21319d" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 40389 } }
I20260806 06:34:55.075347 2411 leader_election.cc:302] T 00000000000000000000000000000000 P 47b57233eaa5474784ee83811d21319d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 47b57233eaa5474784ee83811d21319d; no voters:
I20260806 06:34:55.075461 2411 leader_election.cc:288] T 00000000000000000000000000000000 P 47b57233eaa5474784ee83811d21319d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:34:55.075541 2415 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 47b57233eaa5474784ee83811d21319d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:34:55.075670 2411 sys_catalog.cc:564] T 00000000000000000000000000000000 P 47b57233eaa5474784ee83811d21319d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:34:55.075697 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 2310
I20260806 06:34:55.075728 2415 raft_consensus.cc:686] T 00000000000000000000000000000000 P 47b57233eaa5474784ee83811d21319d [term 1 LEADER]: Becoming Leader. State: Replica: 47b57233eaa5474784ee83811d21319d, State: Running, Role: LEADER
I20260806 06:34:55.075809 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/master-0/wal/instance
I20260806 06:34:55.075882 2415 consensus_queue.cc:237] T 00000000000000000000000000000000 P 47b57233eaa5474784ee83811d21319d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47b57233eaa5474784ee83811d21319d" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 40389 } }
I20260806 06:34:55.075990 7341 external_mini_cluster.cc:1428] Setting key 8e519998f3c53fea0700e984b7b43fa3
I20260806 06:34:55.076256 2417 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47b57233eaa5474784ee83811d21319d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 47b57233eaa5474784ee83811d21319d. Latest consensus state: current_term: 1 leader_uuid: "47b57233eaa5474784ee83811d21319d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47b57233eaa5474784ee83811d21319d" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 40389 } } }
I20260806 06:34:55.076339 2417 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47b57233eaa5474784ee83811d21319d [sys.catalog]: This master's current role is: LEADER
I20260806 06:34:55.076476 2416 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47b57233eaa5474784ee83811d21319d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "47b57233eaa5474784ee83811d21319d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47b57233eaa5474784ee83811d21319d" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 40389 } } }
I20260806 06:34:55.076534 2416 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47b57233eaa5474784ee83811d21319d [sys.catalog]: This master's current role is: LEADER
I20260806 06:34:55.077091 2429 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:34:55.077553 2429 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:34:55.079001 2429 catalog_manager.cc:1347] Generated new cluster ID: d3e6b83852ef4c68ab9e76aa2caa5aa8
I20260806 06:34:55.079054 2429 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:34:55.081442 2433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:55.076711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46644 (local address 127.7.43.126:40389)
0806 06:34:55.077199 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:55.077207 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:55.077293 (+ 86us) server_negotiation.cc:407] Connection header received
0806 06:34:55.077403 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:55.077410 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:55.077473 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:55.077612 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:34:55.077837 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:55.079010 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:55.079785 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:55.080182 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:55.080285 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:55.080851 (+ 566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:34:55.080855 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:34:55.080855 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:34:55.081005 (+ 150us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:34:55.081035 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:34:55.081045 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:34:55.081110 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:55.081178 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 06:34:55.081253 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":77,"threads_started":1}
I20260806 06:34:55.102772 2429 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:34:55.103195 2429 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:34:55.105789 2429 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 47b57233eaa5474784ee83811d21319d: Generated new TSK 0
I20260806 06:34:55.105926 2429 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:34:55.111070 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.65:0
--local_ip_for_outbound_sockets=127.7.43.65
--webserver_interface=127.7.43.65
--webserver_port=0
--tserver_master_addrs=127.7.43.126:40389
--builtin_ntp_servers=127.7.43.84:44207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 06:34:55.209614 2443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:34:55.209793 2443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:34:55.209812 2443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:34:55.210919 2443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:34:55.210956 2443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:34:55.210979 2443 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.65
I20260806 06:34:55.212255 2443 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:44207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.43.65
--webserver_port=0
--tserver_master_addrs=127.7.43.126:40389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2443
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.65
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:34:55.212450 2443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:34:55.212702 2443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:34:55.217211 2453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:34:55.217165 2457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:34:55.217170 2454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:34:55.217197 2443 server_base.cc:1034] running on GCE node
I20260806 06:34:55.217528 2443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:34:55.217748 2443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:34:55.218880 2443 hybrid_clock.cc:648] HybridClock initialized: now 1785998095218868 us; error 29 us; skew 500 ppm
I20260806 06:34:55.220013 2443 webserver.cc:460] Webserver started at http://127.7.43.65:44723/ using document root <none> and password file <none>
I20260806 06:34:55.220221 2443 fs_manager.cc:359] Metadata directory not provided
I20260806 06:34:55.220271 2443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:34:55.220393 2443 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:34:55.221351 2443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/ts-0/data/instance:
uuid: "4dbbedeffc64433bb1a5998b3dd93a32"
format_stamp: "Formatted at 2026-08-06 06:34:55 on dist-test-slave-6zbq"
server_key: "456af4747fa0eafde76ad6298306a3b2"
server_key_iv: "a85bce8a7236eaeb349a4dbce7caa179"
server_key_version: "encryptionkey@0"
I20260806 06:34:55.221662 2443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/ts-0/wal/instance:
uuid: "4dbbedeffc64433bb1a5998b3dd93a32"
format_stamp: "Formatted at 2026-08-06 06:34:55 on dist-test-slave-6zbq"
server_key: "456af4747fa0eafde76ad6298306a3b2"
server_key_iv: "a85bce8a7236eaeb349a4dbce7caa179"
server_key_version: "encryptionkey@0"
I20260806 06:34:55.222894 2443 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:34:55.223555 2466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:55.223704 2443 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:34:55.223784 2443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/ts-0/wal
uuid: "4dbbedeffc64433bb1a5998b3dd93a32"
format_stamp: "Formatted at 2026-08-06 06:34:55 on dist-test-slave-6zbq"
server_key: "456af4747fa0eafde76ad6298306a3b2"
server_key_iv: "a85bce8a7236eaeb349a4dbce7caa179"
server_key_version: "encryptionkey@0"
I20260806 06:34:55.223850 2443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:34:55.231720 2443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:34:55.231947 2443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:34:55.232059 2443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:34:55.232240 2443 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:34:55.232515 2443 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:34:55.232546 2443 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:55.232563 2443 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:34:55.232585 2443 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:55.237857 2443 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.65:46207
I20260806 06:34:55.237910 2638 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.65:46207 every 8 connection(s)
I20260806 06:34:55.238162 2443 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
I20260806 06:34:55.242527 2433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:55.238777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:46713 (local address 127.7.43.126:40389)
0806 06:34:55.238818 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:55.238821 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:55.239035 (+ 214us) server_negotiation.cc:407] Connection header received
0806 06:34:55.239452 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:55.239453 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:55.239464 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:55.239498 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:34:55.240098 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:55.240564 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:55.241431 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:55.241599 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:55.241640 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:55.242326 (+ 686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:34:55.242327 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:34:55.242327 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:34:55.242331 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:34:55.242356 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:34:55.242359 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:34:55.242390 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:55.242453 (+ 63us) server_negotiation.cc:299] Negotiation successful
0806 06:34:55.242472 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:34:55.242704 2642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:55.238763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40389 (local address 127.7.43.65:46713)
0806 06:34:55.238930 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:55.238960 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:55.239369 (+ 409us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:55.239551 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:55.239554 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:55.239562 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:34:55.240039 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:55.240044 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:55.240620 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:55.240621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:55.241374 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:55.241376 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:55.241648 (+ 272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:55.241649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:55.241768 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:55.242218 (+ 450us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:34:55.242218 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:34:55.242245 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:34:55.242267 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:34:55.242393 (+ 126us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:34:55.242396 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:34:55.242399 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:34:55.242452 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 06:34:55.242525 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":82,"threads_started":1}
I20260806 06:34:55.243149 2640 heartbeater.cc:347] Connected to a master server at 127.7.43.126:40389
I20260806 06:34:55.243222 2640 heartbeater.cc:464] Registering TS with master...
I20260806 06:34:55.243382 2640 heartbeater.cc:511] Master 127.7.43.126:40389 requested a full tablet report, sending...
I20260806 06:34:55.243832 2359 ts_manager.cc:194] Registered new tserver with Master: 4dbbedeffc64433bb1a5998b3dd93a32 (127.7.43.65:46207)
I20260806 06:34:55.244387 2359 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.43.65:46713
I20260806 06:34:55.246850 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 2443
I20260806 06:34:55.246951 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/ts-0/wal/instance
I20260806 06:34:55.247120 7341 external_mini_cluster.cc:1428] Setting key 6f40de5e558ac0d7cd40fc03a92c8998
I20260806 06:34:55.247900 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.66:0
--local_ip_for_outbound_sockets=127.7.43.66
--webserver_interface=127.7.43.66
--webserver_port=0
--tserver_master_addrs=127.7.43.126:40389
--builtin_ntp_servers=127.7.43.84:44207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 06:34:55.325263 2644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:34:55.325433 2644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:34:55.325452 2644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:34:55.326586 2644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:34:55.326622 2644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:34:55.326649 2644 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.66
I20260806 06:34:55.327965 2644 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:44207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.43.66
--webserver_port=0
--tserver_master_addrs=127.7.43.126:40389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2644
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.66
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:34:55.328156 2644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:34:55.328406 2644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:34:55.332854 2655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:34:55.332911 2653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:34:55.332984 2651 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:34:55.333002 2644 server_base.cc:1034] running on GCE node
I20260806 06:34:55.333237 2644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:34:55.333454 2644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:34:55.334582 2644 hybrid_clock.cc:648] HybridClock initialized: now 1785998095334567 us; error 27 us; skew 500 ppm
I20260806 06:34:55.335711 2644 webserver.cc:460] Webserver started at http://127.7.43.66:45291/ using document root <none> and password file <none>
I20260806 06:34:55.335911 2644 fs_manager.cc:359] Metadata directory not provided
I20260806 06:34:55.335956 2644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:34:55.336077 2644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:34:55.337005 2644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/ts-1/data/instance:
uuid: "1c143d8a62f14f50bd97ed08f3698200"
format_stamp: "Formatted at 2026-08-06 06:34:55 on dist-test-slave-6zbq"
server_key: "40cf84edec5378269dce3c49f4b6e622"
server_key_iv: "3e34335c5f647a767c86f7f788a01d26"
server_key_version: "encryptionkey@0"
I20260806 06:34:55.337311 2644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/ts-1/wal/instance:
uuid: "1c143d8a62f14f50bd97ed08f3698200"
format_stamp: "Formatted at 2026-08-06 06:34:55 on dist-test-slave-6zbq"
server_key: "40cf84edec5378269dce3c49f4b6e622"
server_key_iv: "3e34335c5f647a767c86f7f788a01d26"
server_key_version: "encryptionkey@0"
I20260806 06:34:55.338438 2644 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:34:55.339066 2664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:55.339218 2644 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:34:55.339298 2644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/ts-1/wal
uuid: "1c143d8a62f14f50bd97ed08f3698200"
format_stamp: "Formatted at 2026-08-06 06:34:55 on dist-test-slave-6zbq"
server_key: "40cf84edec5378269dce3c49f4b6e622"
server_key_iv: "3e34335c5f647a767c86f7f788a01d26"
server_key_version: "encryptionkey@0"
I20260806 06:34:55.339360 2644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:34:55.346787 2644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:34:55.347002 2644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:34:55.347110 2644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:34:55.347308 2644 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:34:55.347569 2644 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:34:55.347600 2644 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:55.347625 2644 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:34:55.347649 2644 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:55.352759 2644 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.66:41427
I20260806 06:34:55.352790 2836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.66:41427 every 8 connection(s)
I20260806 06:34:55.353044 2644 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
I20260806 06:34:55.357204 2433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:55.353604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:37057 (local address 127.7.43.126:40389)
0806 06:34:55.353661 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:55.353663 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:55.353859 (+ 196us) server_negotiation.cc:407] Connection header received
0806 06:34:55.354260 (+ 401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:55.354261 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:55.354270 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:55.354318 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:34:55.354905 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:55.355380 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:55.356236 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:55.356411 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:55.356452 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:55.357017 (+ 565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:34:55.357018 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:34:55.357019 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:34:55.357022 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:34:55.357036 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:34:55.357041 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:34:55.357076 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:55.357126 (+ 50us) server_negotiation.cc:299] Negotiation successful
0806 06:34:55.357149 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 06:34:55.357378 2839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:55.353585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40389 (local address 127.7.43.66:37057)
0806 06:34:55.353757 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:55.353791 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:55.354185 (+ 394us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:55.354392 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:55.354396 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:55.354406 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:34:55.354842 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:55.354846 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:55.355437 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:55.355437 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:55.356182 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:55.356184 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:55.356460 (+ 276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:55.356460 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:55.356553 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:55.356907 (+ 354us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:34:55.356908 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:34:55.356931 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:34:55.356952 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:34:55.357077 (+ 125us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:34:55.357079 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:34:55.357081 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:34:55.357121 (+ 40us) client_negotiation.cc:240] Negotiation successful
0806 06:34:55.357199 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":87,"threads_started":1}
I20260806 06:34:55.357796 2837 heartbeater.cc:347] Connected to a master server at 127.7.43.126:40389
I20260806 06:34:55.357872 2837 heartbeater.cc:464] Registering TS with master...
I20260806 06:34:55.358019 2837 heartbeater.cc:511] Master 127.7.43.126:40389 requested a full tablet report, sending...
I20260806 06:34:55.358326 2359 ts_manager.cc:194] Registered new tserver with Master: 1c143d8a62f14f50bd97ed08f3698200 (127.7.43.66:41427)
I20260806 06:34:55.358676 2359 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.43.66:37057
I20260806 06:34:55.362555 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 2644
I20260806 06:34:55.362638 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/ts-1/wal/instance
I20260806 06:34:55.362778 7341 external_mini_cluster.cc:1428] Setting key 6ae5aec7c679520cb7e41663de9ccc08
I20260806 06:34:55.363474 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.67:0
--local_ip_for_outbound_sockets=127.7.43.67
--webserver_interface=127.7.43.67
--webserver_port=0
--tserver_master_addrs=127.7.43.126:40389
--builtin_ntp_servers=127.7.43.84:44207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 06:34:55.440197 2841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:34:55.440366 2841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:34:55.440392 2841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:34:55.441509 2841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:34:55.441546 2841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:34:55.441578 2841 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.67
I20260806 06:34:55.442932 2841 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:44207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.43.67
--webserver_port=0
--tserver_master_addrs=127.7.43.126:40389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2841
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.67
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:34:55.443135 2841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:34:55.443380 2841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:34:55.447830 2850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:34:55.447836 2848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:34:55.447863 2841 server_base.cc:1034] running on GCE node
W20260806 06:34:55.447815 2852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:34:55.448167 2841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:34:55.448405 2841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:34:55.449532 2841 hybrid_clock.cc:648] HybridClock initialized: now 1785998095449521 us; error 25 us; skew 500 ppm
I20260806 06:34:55.450567 2841 webserver.cc:460] Webserver started at http://127.7.43.67:38791/ using document root <none> and password file <none>
I20260806 06:34:55.450775 2841 fs_manager.cc:359] Metadata directory not provided
I20260806 06:34:55.450817 2841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:34:55.450932 2841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:34:55.451855 2841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/ts-2/data/instance:
uuid: "de8a97824f5a4040b0a5cf456b22dfb6"
format_stamp: "Formatted at 2026-08-06 06:34:55 on dist-test-slave-6zbq"
server_key: "32794198cb2bfc3e0da7cf264a45418e"
server_key_iv: "9e13c307fcbe790b25976121111a023a"
server_key_version: "encryptionkey@0"
I20260806 06:34:55.452157 2841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/ts-2/wal/instance:
uuid: "de8a97824f5a4040b0a5cf456b22dfb6"
format_stamp: "Formatted at 2026-08-06 06:34:55 on dist-test-slave-6zbq"
server_key: "32794198cb2bfc3e0da7cf264a45418e"
server_key_iv: "9e13c307fcbe790b25976121111a023a"
server_key_version: "encryptionkey@0"
I20260806 06:34:55.453243 2841 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:34:55.453805 2862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:55.453931 2841 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:34:55.453999 2841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/ts-2/wal
uuid: "de8a97824f5a4040b0a5cf456b22dfb6"
format_stamp: "Formatted at 2026-08-06 06:34:55 on dist-test-slave-6zbq"
server_key: "32794198cb2bfc3e0da7cf264a45418e"
server_key_iv: "9e13c307fcbe790b25976121111a023a"
server_key_version: "encryptionkey@0"
I20260806 06:34:55.454061 2841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:34:55.471199 2841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:34:55.471421 2841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:34:55.471534 2841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:34:55.471720 2841 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:34:55.471988 2841 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:34:55.472020 2841 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:55.472046 2841 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:34:55.472069 2841 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:55.477485 2841 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.67:42789
I20260806 06:34:55.477521 3035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.67:42789 every 8 connection(s)
I20260806 06:34:55.477774 2841 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
I20260806 06:34:55.477938 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 2841
I20260806 06:34:55.478031 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/ts-2/wal/instance
I20260806 06:34:55.478220 7341 external_mini_cluster.cc:1428] Setting key 18536bb2e101d614278de50c606f6ba4
I20260806 06:34:55.482141 2433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:55.478399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:43843 (local address 127.7.43.126:40389)
0806 06:34:55.478460 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:55.478462 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:55.478689 (+ 227us) server_negotiation.cc:407] Connection header received
0806 06:34:55.479168 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:55.479169 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:55.479179 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:55.479221 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:34:55.479776 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:55.480249 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:55.481102 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:55.481275 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:55.481314 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:55.481976 (+ 662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:34:55.481977 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:34:55.481977 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:34:55.481981 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:34:55.481990 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:34:55.481992 (+ 2us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:34:55.482021 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:55.482070 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 06:34:55.482087 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 06:34:55.482353 3039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:55.478399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:40389 (local address 127.7.43.67:43843)
0806 06:34:55.478584 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:55.478623 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:55.479101 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:55.479275 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:55.479278 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:55.479286 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:34:55.479716 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:55.479720 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:55.480306 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:55.480306 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:55.481034 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:55.481036 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:55.481322 (+ 286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:55.481322 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:55.481417 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:55.481868 (+ 451us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:34:55.481868 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:34:55.481893 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:34:55.481914 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:34:55.482022 (+ 108us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:34:55.482025 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:34:55.482027 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:34:55.482068 (+ 41us) client_negotiation.cc:240] Negotiation successful
0806 06:34:55.482140 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":88,"threads_started":1}
I20260806 06:34:55.482738 3037 heartbeater.cc:347] Connected to a master server at 127.7.43.126:40389
I20260806 06:34:55.482816 3037 heartbeater.cc:464] Registering TS with master...
I20260806 06:34:55.482969 3037 heartbeater.cc:511] Master 127.7.43.126:40389 requested a full tablet report, sending...
I20260806 06:34:55.483304 2359 ts_manager.cc:194] Registered new tserver with Master: de8a97824f5a4040b0a5cf456b22dfb6 (127.7.43.67:42789)
I20260806 06:34:55.483639 2359 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.43.67:43843
I20260806 06:34:55.489673 7341 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:34:55.493124 2433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:55.490869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46658 (local address 127.7.43.126:40389)
0806 06:34:55.490932 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:55.490934 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:55.490976 (+ 42us) server_negotiation.cc:407] Connection header received
0806 06:34:55.491032 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:55.491033 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:55.491041 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:55.491074 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:34:55.491189 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:55.491610 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:55.492256 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:55.492416 (+ 160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:55.492455 (+ 39us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:55.492921 (+ 466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:34:55.492922 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:34:55.492922 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:34:55.492926 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:34:55.492939 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:34:55.492942 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:34:55.493004 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:55.493048 (+ 44us) server_negotiation.cc:299] Negotiation successful
0806 06:34:55.493057 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 06:34:55.494484 2359 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46658:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:34:55.495885 2359 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:34:55.501367 3048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:55.497645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:46207 (local address 127.0.0.1:40302)
0806 06:34:55.497746 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:55.497774 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:55.497832 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:55.498113 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:55.498116 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:55.498118 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:34:55.498186 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:55.498188 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:55.499437 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:55.499438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:55.500381 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:55.500385 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:55.501079 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:55.501080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:55.501170 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:55.501184 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 06:34:55.501251 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 06:34:55.501299 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79,"mutex_wait_us":32,"thread_start_us":44,"threads_started":1}
I20260806 06:34:55.501377 3049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:55.497755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40302 (local address 127.7.43.65:46207)
0806 06:34:55.497953 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:55.497957 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:55.497967 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:34:55.498008 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:55.498010 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:55.498025 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:55.498104 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:34:55.498276 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:55.499348 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:55.500465 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:55.501003 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:55.501065 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:55.501161 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:55.501266 (+ 105us) server_negotiation.cc:299] Negotiation successful
0806 06:34:55.501299 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":81,"threads_started":1}
I20260806 06:34:55.501588 3051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:55.497812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.67:42789 (local address 127.0.0.1:47008)
0806 06:34:55.498098 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:55.498107 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:55.498151 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:55.498356 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:55.498356 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:55.498358 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:34:55.498401 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:55.498402 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:55.499404 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:55.499405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:55.500313 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:55.500315 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:55.501325 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:55.501326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:55.501385 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:55.501389 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:34:55.501484 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 06:34:55.501519 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":70,"threads_started":1}
I20260806 06:34:55.501869 3052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:55.497885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47008 (local address 127.7.43.67:42789)
0806 06:34:55.498038 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:55.498042 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:55.498207 (+ 165us) server_negotiation.cc:407] Connection header received
0806 06:34:55.498255 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:55.498258 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:55.498271 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:55.498365 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:34:55.498505 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:55.499327 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:55.500470 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:55.501158 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:55.501214 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:55.501291 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:55.501766 (+ 475us) server_negotiation.cc:299] Negotiation successful
0806 06:34:55.501802 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":79,"threads_started":1}
I20260806 06:34:55.502276 2536 tablet_service.cc:1467] Processing CreateTablet for tablet 307111a1c1db48a48ea2bdbe8201c64b (DEFAULT_TABLE table=test-table [id=383c086b7a4644989c794c8961e7d201]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:34:55.502596 3053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:55.497970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.66:41427 (local address 127.0.0.1:46526)
0806 06:34:55.498396 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:55.498406 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:55.498455 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:55.498807 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:55.498808 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:55.498810 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:34:55.498848 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:55.498850 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:55.500032 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:55.500033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:55.501058 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:55.501061 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:55.502406 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:55.502407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:55.502455 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:55.502458 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:34:55.502508 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 06:34:55.502547 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":62,"threads_started":1}
I20260806 06:34:55.502511 2941 tablet_service.cc:1467] Processing CreateTablet for tablet 307111a1c1db48a48ea2bdbe8201c64b (DEFAULT_TABLE table=test-table [id=383c086b7a4644989c794c8961e7d201]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:34:55.502696 2536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 307111a1c1db48a48ea2bdbe8201c64b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:34:55.502781 2941 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 307111a1c1db48a48ea2bdbe8201c64b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:34:55.502861 3054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:55.498083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46526 (local address 127.7.43.66:41427)
0806 06:34:55.498262 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:55.498267 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:55.498508 (+ 241us) server_negotiation.cc:407] Connection header received
0806 06:34:55.498647 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:55.498650 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:55.498665 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:55.498742 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:34:55.498917 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:55.499948 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:55.501172 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:55.501856 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:55.501930 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:55.502015 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:55.502737 (+ 722us) server_negotiation.cc:299] Negotiation successful
0806 06:34:55.502779 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":96,"threads_started":1}
I20260806 06:34:55.503366 2729 tablet_service.cc:1467] Processing CreateTablet for tablet 307111a1c1db48a48ea2bdbe8201c64b (DEFAULT_TABLE table=test-table [id=383c086b7a4644989c794c8961e7d201]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:34:55.503598 2729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 307111a1c1db48a48ea2bdbe8201c64b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:34:55.505951 3056 tablet_bootstrap.cc:492] T 307111a1c1db48a48ea2bdbe8201c64b P de8a97824f5a4040b0a5cf456b22dfb6: Bootstrap starting.
I20260806 06:34:55.506450 3055 tablet_bootstrap.cc:492] T 307111a1c1db48a48ea2bdbe8201c64b P 4dbbedeffc64433bb1a5998b3dd93a32: Bootstrap starting.
I20260806 06:34:55.506561 3056 tablet_bootstrap.cc:654] T 307111a1c1db48a48ea2bdbe8201c64b P de8a97824f5a4040b0a5cf456b22dfb6: Neither blocks nor log segments found. Creating new log.
I20260806 06:34:55.506836 3056 log.cc:824] T 307111a1c1db48a48ea2bdbe8201c64b P de8a97824f5a4040b0a5cf456b22dfb6: Log is configured to *not* fsync() on all Append() calls
I20260806 06:34:55.506999 3055 tablet_bootstrap.cc:654] T 307111a1c1db48a48ea2bdbe8201c64b P 4dbbedeffc64433bb1a5998b3dd93a32: Neither blocks nor log segments found. Creating new log.
I20260806 06:34:55.507248 3055 log.cc:824] T 307111a1c1db48a48ea2bdbe8201c64b P 4dbbedeffc64433bb1a5998b3dd93a32: Log is configured to *not* fsync() on all Append() calls
I20260806 06:34:55.507443 3056 tablet_bootstrap.cc:492] T 307111a1c1db48a48ea2bdbe8201c64b P de8a97824f5a4040b0a5cf456b22dfb6: No bootstrap required, opened a new log
I20260806 06:34:55.507515 3056 ts_tablet_manager.cc:1397] T 307111a1c1db48a48ea2bdbe8201c64b P de8a97824f5a4040b0a5cf456b22dfb6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:34:55.507547 3057 tablet_bootstrap.cc:492] T 307111a1c1db48a48ea2bdbe8201c64b P 1c143d8a62f14f50bd97ed08f3698200: Bootstrap starting.
I20260806 06:34:55.507978 3055 tablet_bootstrap.cc:492] T 307111a1c1db48a48ea2bdbe8201c64b P 4dbbedeffc64433bb1a5998b3dd93a32: No bootstrap required, opened a new log
I20260806 06:34:55.508037 3055 ts_tablet_manager.cc:1397] T 307111a1c1db48a48ea2bdbe8201c64b P 4dbbedeffc64433bb1a5998b3dd93a32: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:34:55.508157 3057 tablet_bootstrap.cc:654] T 307111a1c1db48a48ea2bdbe8201c64b P 1c143d8a62f14f50bd97ed08f3698200: Neither blocks nor log segments found. Creating new log.
I20260806 06:34:55.508430 3057 log.cc:824] T 307111a1c1db48a48ea2bdbe8201c64b P 1c143d8a62f14f50bd97ed08f3698200: Log is configured to *not* fsync() on all Append() calls
I20260806 06:34:55.509119 3057 tablet_bootstrap.cc:492] T 307111a1c1db48a48ea2bdbe8201c64b P 1c143d8a62f14f50bd97ed08f3698200: No bootstrap required, opened a new log
I20260806 06:34:55.509173 3057 ts_tablet_manager.cc:1397] T 307111a1c1db48a48ea2bdbe8201c64b P 1c143d8a62f14f50bd97ed08f3698200: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:34:55.509554 3056 raft_consensus.cc:348] T 307111a1c1db48a48ea2bdbe8201c64b P de8a97824f5a4040b0a5cf456b22dfb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbbedeffc64433bb1a5998b3dd93a32" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 46207 } } peers { permanent_uuid: "de8a97824f5a4040b0a5cf456b22dfb6" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 42789 } } peers { permanent_uuid: "1c143d8a62f14f50bd97ed08f3698200" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 41427 } }
I20260806 06:34:55.509704 3056 raft_consensus.cc:374] T 307111a1c1db48a48ea2bdbe8201c64b P de8a97824f5a4040b0a5cf456b22dfb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:34:55.509735 3056 raft_consensus.cc:729] T 307111a1c1db48a48ea2bdbe8201c64b P de8a97824f5a4040b0a5cf456b22dfb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de8a97824f5a4040b0a5cf456b22dfb6, State: Initialized, Role: FOLLOWER
I20260806 06:34:55.509864 3056 consensus_queue.cc:260] T 307111a1c1db48a48ea2bdbe8201c64b P de8a97824f5a4040b0a5cf456b22dfb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbbedeffc64433bb1a5998b3dd93a32" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 46207 } } peers { permanent_uuid: "de8a97824f5a4040b0a5cf456b22dfb6" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 42789 } } peers { permanent_uuid: "1c143d8a62f14f50bd97ed08f3698200" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 41427 } }
I20260806 06:34:55.510109 3056 ts_tablet_manager.cc:1428] T 307111a1c1db48a48ea2bdbe8201c64b P de8a97824f5a4040b0a5cf456b22dfb6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:34:55.510020 3055 raft_consensus.cc:348] T 307111a1c1db48a48ea2bdbe8201c64b P 4dbbedeffc64433bb1a5998b3dd93a32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbbedeffc64433bb1a5998b3dd93a32" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 46207 } } peers { permanent_uuid: "de8a97824f5a4040b0a5cf456b22dfb6" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 42789 } } peers { permanent_uuid: "1c143d8a62f14f50bd97ed08f3698200" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 41427 } }
I20260806 06:34:55.510146 3037 heartbeater.cc:503] Master 127.7.43.126:40389 was elected leader, sending a full tablet report...
I20260806 06:34:55.510166 3055 raft_consensus.cc:374] T 307111a1c1db48a48ea2bdbe8201c64b P 4dbbedeffc64433bb1a5998b3dd93a32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:34:55.510215 3055 raft_consensus.cc:729] T 307111a1c1db48a48ea2bdbe8201c64b P 4dbbedeffc64433bb1a5998b3dd93a32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dbbedeffc64433bb1a5998b3dd93a32, State: Initialized, Role: FOLLOWER
I20260806 06:34:55.510313 3055 consensus_queue.cc:260] T 307111a1c1db48a48ea2bdbe8201c64b P 4dbbedeffc64433bb1a5998b3dd93a32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbbedeffc64433bb1a5998b3dd93a32" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 46207 } } peers { permanent_uuid: "de8a97824f5a4040b0a5cf456b22dfb6" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 42789 } } peers { permanent_uuid: "1c143d8a62f14f50bd97ed08f3698200" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 41427 } }
I20260806 06:34:55.510576 3055 ts_tablet_manager.cc:1428] T 307111a1c1db48a48ea2bdbe8201c64b P 4dbbedeffc64433bb1a5998b3dd93a32: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 06:34:55.510596 2640 heartbeater.cc:503] Master 127.7.43.126:40389 was elected leader, sending a full tablet report...
I20260806 06:34:55.511513 3057 raft_consensus.cc:348] T 307111a1c1db48a48ea2bdbe8201c64b P 1c143d8a62f14f50bd97ed08f3698200 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbbedeffc64433bb1a5998b3dd93a32" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 46207 } } peers { permanent_uuid: "de8a97824f5a4040b0a5cf456b22dfb6" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 42789 } } peers { permanent_uuid: "1c143d8a62f14f50bd97ed08f3698200" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 41427 } }
I20260806 06:34:55.511641 3057 raft_consensus.cc:374] T 307111a1c1db48a48ea2bdbe8201c64b P 1c143d8a62f14f50bd97ed08f3698200 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:34:55.511663 3057 raft_consensus.cc:729] T 307111a1c1db48a48ea2bdbe8201c64b P 1c143d8a62f14f50bd97ed08f3698200 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c143d8a62f14f50bd97ed08f3698200, State: Initialized, Role: FOLLOWER
I20260806 06:34:55.511770 3057 consensus_queue.cc:260] T 307111a1c1db48a48ea2bdbe8201c64b P 1c143d8a62f14f50bd97ed08f3698200 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbbedeffc64433bb1a5998b3dd93a32" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 46207 } } peers { permanent_uuid: "de8a97824f5a4040b0a5cf456b22dfb6" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 42789 } } peers { permanent_uuid: "1c143d8a62f14f50bd97ed08f3698200" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 41427 } }
I20260806 06:34:55.511976 3057 ts_tablet_manager.cc:1428] T 307111a1c1db48a48ea2bdbe8201c64b P 1c143d8a62f14f50bd97ed08f3698200: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 06:34:55.512013 2837 heartbeater.cc:503] Master 127.7.43.126:40389 was elected leader, sending a full tablet report...
W20260806 06:34:55.603484 2838 tablet.cc:2369] T 307111a1c1db48a48ea2bdbe8201c64b P 1c143d8a62f14f50bd97ed08f3698200: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:34:55.660051 3063 raft_consensus.cc:482] T 307111a1c1db48a48ea2bdbe8201c64b P 4dbbedeffc64433bb1a5998b3dd93a32 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:34:55.660164 3063 raft_consensus.cc:504] T 307111a1c1db48a48ea2bdbe8201c64b P 4dbbedeffc64433bb1a5998b3dd93a32 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbbedeffc64433bb1a5998b3dd93a32" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 46207 } } peers { permanent_uuid: "de8a97824f5a4040b0a5cf456b22dfb6" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 42789 } } peers { permanent_uuid: "1c143d8a62f14f50bd97ed08f3698200" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 41427 } }
I20260806 06:34:55.660516 3063 leader_election.cc:288] T 307111a1c1db48a48ea2bdbe8201c64b P 4dbbedeffc64433bb1a5998b3dd93a32 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de8a97824f5a4040b0a5cf456b22dfb6 (127.7.43.67:42789), 1c143d8a62f14f50bd97ed08f3698200 (127.7.43.66:41427)
I20260806 06:34:55.664199 3054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:55.660863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:43647 (local address 127.7.43.66:41427)
0806 06:34:55.660920 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:55.660922 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:55.660990 (+ 68us) server_negotiation.cc:407] Connection header received
0806 06:34:55.661048 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:55.661049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:55.661055 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:55.661110 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:34:55.661231 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:55.661910 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:55.663296 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:55.663850 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:55.663922 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:55.663931 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:55.664095 (+ 164us) server_negotiation.cc:299] Negotiation successful
0806 06:34:55.664130 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 06:34:55.664201 3052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:55.660726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:46283 (local address 127.7.43.67:42789)
0806 06:34:55.660786 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:55.660788 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:55.660799 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:34:55.660889 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:55.660890 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:55.660901 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:55.660951 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:34:55.661135 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:55.661877 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:55.663300 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:55.663850 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:55.663922 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:55.663932 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:55.664094 (+ 162us) server_negotiation.cc:299] Negotiation successful
0806 06:34:55.664133 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 06:34:55.664238 2642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:55.660665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.67:42789 (local address 127.7.43.65:46283)
0806 06:34:55.660730 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:55.660743 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:55.660795 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:55.660979 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:55.660980 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:55.660982 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:34:55.661056 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:55.661058 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:55.661956 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:55.661957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:55.663221 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:55.663224 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:55.663940 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:55.663941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:55.664029 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:55.664033 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:34:55.664085 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 06:34:55.664137 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":34}
I20260806 06:34:55.664242 3066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:55.660775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.66:41427 (local address 127.7.43.65:43647)
0806 06:34:55.660927 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:55.660936 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:55.660992 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:55.661122 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:55.661123 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:55.661124 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:34:55.661179 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:55.661180 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:55.661971 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:55.661972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:55.663221 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:55.663224 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:55.663926 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:55.663927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:55.664029 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:55.664034 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:34:55.664085 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 06:34:55.664134 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":51,"thread_start_us":93,"threads_started":1}
I20260806 06:34:55.664559 2974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "307111a1c1db48a48ea2bdbe8201c64b" candidate_uuid: "4dbbedeffc64433bb1a5998b3dd93a32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de8a97824f5a4040b0a5cf456b22dfb6" is_pre_election: true
I20260806 06:34:55.664686 2974 raft_consensus.cc:2454] T 307111a1c1db48a48ea2bdbe8201c64b P de8a97824f5a4040b0a5cf456b22dfb6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dbbedeffc64433bb1a5998b3dd93a32 in term 0.
I20260806 06:34:55.664918 2468 leader_election.cc:302] T 307111a1c1db48a48ea2bdbe8201c64b P 4dbbedeffc64433bb1a5998b3dd93a32 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dbbedeffc64433bb1a5998b3dd93a32, de8a97824f5a4040b0a5cf456b22dfb6; no voters:
I20260806 06:34:55.664952 2759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "307111a1c1db48a48ea2bdbe8201c64b" candidate_uuid: "4dbbedeffc64433bb1a5998b3dd93a32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c143d8a62f14f50bd97ed08f3698200" is_pre_election: true
I20260806 06:34:55.665081 2759 raft_consensus.cc:2454] T 307111a1c1db48a48ea2bdbe8201c64b P 1c143d8a62f14f50bd97ed08f3698200 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dbbedeffc64433bb1a5998b3dd93a32 in term 0.
I20260806 06:34:55.665062 3063 raft_consensus.cc:2781] T 307111a1c1db48a48ea2bdbe8201c64b P 4dbbedeffc64433bb1a5998b3dd93a32 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:34:55.665122 3063 raft_consensus.cc:482] T 307111a1c1db48a48ea2bdbe8201c64b P 4dbbedeffc64433bb1a5998b3dd93a32 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:34:55.665145 3063 raft_consensus.cc:3037] T 307111a1c1db48a48ea2bdbe8201c64b P 4dbbedeffc64433bb1a5998b3dd93a32 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:34:55.665854 3063 raft_consensus.cc:504] T 307111a1c1db48a48ea2bdbe8201c64b P 4dbbedeffc64433bb1a5998b3dd93a32 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbbedeffc64433bb1a5998b3dd93a32" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 46207 } } peers { permanent_uuid: "de8a97824f5a4040b0a5cf456b22dfb6" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 42789 } } peers { permanent_uuid: "1c143d8a62f14f50bd97ed08f3698200" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 41427 } }
I20260806 06:34:55.665966 3063 leader_election.cc:288] T 307111a1c1db48a48ea2bdbe8201c64b P 4dbbedeffc64433bb1a5998b3dd93a32 [CANDIDATE]: Term 1 election: Requested vote from peers de8a97824f5a4040b0a5cf456b22dfb6 (127.7.43.67:42789), 1c143d8a62f14f50bd97ed08f3698200 (127.7.43.66:41427)
I20260806 06:34:55.666131 2759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "307111a1c1db48a48ea2bdbe8201c64b" candidate_uuid: "4dbbedeffc64433bb1a5998b3dd93a32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c143d8a62f14f50bd97ed08f3698200"
I20260806 06:34:55.666147 2974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "307111a1c1db48a48ea2bdbe8201c64b" candidate_uuid: "4dbbedeffc64433bb1a5998b3dd93a32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de8a97824f5a4040b0a5cf456b22dfb6"
I20260806 06:34:55.666203 2759 raft_consensus.cc:3037] T 307111a1c1db48a48ea2bdbe8201c64b P 1c143d8a62f14f50bd97ed08f3698200 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:34:55.666220 2974 raft_consensus.cc:3037] T 307111a1c1db48a48ea2bdbe8201c64b P de8a97824f5a4040b0a5cf456b22dfb6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:34:55.666901 2759 raft_consensus.cc:2454] T 307111a1c1db48a48ea2bdbe8201c64b P 1c143d8a62f14f50bd97ed08f3698200 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dbbedeffc64433bb1a5998b3dd93a32 in term 1.
I20260806 06:34:55.666916 2974 raft_consensus.cc:2454] T 307111a1c1db48a48ea2bdbe8201c64b P de8a97824f5a4040b0a5cf456b22dfb6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dbbedeffc64433bb1a5998b3dd93a32 in term 1.
I20260806 06:34:55.667057 2468 leader_election.cc:302] T 307111a1c1db48a48ea2bdbe8201c64b P 4dbbedeffc64433bb1a5998b3dd93a32 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c143d8a62f14f50bd97ed08f3698200, 4dbbedeffc64433bb1a5998b3dd93a32; no voters:
I20260806 06:34:55.667138 3063 raft_consensus.cc:2781] T 307111a1c1db48a48ea2bdbe8201c64b P 4dbbedeffc64433bb1a5998b3dd93a32 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:34:55.667273 3063 raft_consensus.cc:686] T 307111a1c1db48a48ea2bdbe8201c64b P 4dbbedeffc64433bb1a5998b3dd93a32 [term 1 LEADER]: Becoming Leader. State: Replica: 4dbbedeffc64433bb1a5998b3dd93a32, State: Running, Role: LEADER
I20260806 06:34:55.667371 3063 consensus_queue.cc:237] T 307111a1c1db48a48ea2bdbe8201c64b P 4dbbedeffc64433bb1a5998b3dd93a32 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbbedeffc64433bb1a5998b3dd93a32" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 46207 } } peers { permanent_uuid: "de8a97824f5a4040b0a5cf456b22dfb6" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 42789 } } peers { permanent_uuid: "1c143d8a62f14f50bd97ed08f3698200" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 41427 } }
I20260806 06:34:55.668434 2358 catalog_manager.cc:5495] T 307111a1c1db48a48ea2bdbe8201c64b P 4dbbedeffc64433bb1a5998b3dd93a32 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4dbbedeffc64433bb1a5998b3dd93a32 (127.7.43.65). New cstate: current_term: 1 leader_uuid: "4dbbedeffc64433bb1a5998b3dd93a32" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbbedeffc64433bb1a5998b3dd93a32" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 46207 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de8a97824f5a4040b0a5cf456b22dfb6" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 42789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c143d8a62f14f50bd97ed08f3698200" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 41427 } health_report { overall_health: UNKNOWN } } }
I20260806 06:34:55.682937 3049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:55.680727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40310 (local address 127.7.43.65:46207)
0806 06:34:55.680774 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:55.680776 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:55.680783 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:34:55.680843 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:55.680843 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:55.680860 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:55.680918 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:34:55.681031 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:55.681476 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:55.682196 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:55.682411 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:55.682465 (+ 54us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:55.682612 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:34:55.682705 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:34:55.682788 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:55.682839 (+ 51us) server_negotiation.cc:299] Negotiation successful
0806 06:34:55.682848 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 06:34:55.684551 2759 raft_consensus.cc:1261] T 307111a1c1db48a48ea2bdbe8201c64b P 1c143d8a62f14f50bd97ed08f3698200 [term 1 FOLLOWER]: Refusing update from remote peer 4dbbedeffc64433bb1a5998b3dd93a32: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:34:55.684551 2974 raft_consensus.cc:1261] T 307111a1c1db48a48ea2bdbe8201c64b P de8a97824f5a4040b0a5cf456b22dfb6 [term 1 FOLLOWER]: Refusing update from remote peer 4dbbedeffc64433bb1a5998b3dd93a32: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:34:55.684872 3063 consensus_queue.cc:1035] T 307111a1c1db48a48ea2bdbe8201c64b P 4dbbedeffc64433bb1a5998b3dd93a32 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c143d8a62f14f50bd97ed08f3698200" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 41427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:34:55.685005 3067 consensus_queue.cc:1035] T 307111a1c1db48a48ea2bdbe8201c64b P 4dbbedeffc64433bb1a5998b3dd93a32 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de8a97824f5a4040b0a5cf456b22dfb6" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 42789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:34:55.687074 3070 mvcc.cc:205] Tried to move back new op lower bound from 7315448199921385472 to 7315448199854092288. Current Snapshot: MvccSnapshot[applied={T|T < 7315448199921385472}]
I20260806 06:34:55.689493 2359 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46658:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:34:55.689568 2359 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46658:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:34:55.690207 2359 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 47b57233eaa5474784ee83811d21319d: Sending DeleteTablet for 3 replicas of tablet 307111a1c1db48a48ea2bdbe8201c64b
I20260806 06:34:55.690570 2536 tablet_service.cc:1514] Processing DeleteTablet for tablet 307111a1c1db48a48ea2bdbe8201c64b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:34:55 UTC) from {username='slave'} at 127.0.0.1:40302
I20260806 06:34:55.690608 2941 tablet_service.cc:1514] Processing DeleteTablet for tablet 307111a1c1db48a48ea2bdbe8201c64b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:34:55 UTC) from {username='slave'} at 127.0.0.1:47008
I20260806 06:34:55.690630 2729 tablet_service.cc:1514] Processing DeleteTablet for tablet 307111a1c1db48a48ea2bdbe8201c64b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:34:55 UTC) from {username='slave'} at 127.0.0.1:46526
I20260806 06:34:55.690879 3087 tablet_replica.cc:330] T 307111a1c1db48a48ea2bdbe8201c64b P 4dbbedeffc64433bb1a5998b3dd93a32: stopping tablet replica
I20260806 06:34:55.690954 3088 tablet_replica.cc:330] T 307111a1c1db48a48ea2bdbe8201c64b P 1c143d8a62f14f50bd97ed08f3698200: stopping tablet replica
I20260806 06:34:55.690971 3087 raft_consensus.cc:2229] T 307111a1c1db48a48ea2bdbe8201c64b P 4dbbedeffc64433bb1a5998b3dd93a32 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:34:55.691047 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 2443
I20260806 06:34:55.691056 3088 raft_consensus.cc:2229] T 307111a1c1db48a48ea2bdbe8201c64b P 1c143d8a62f14f50bd97ed08f3698200 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:34:55.691071 3087 raft_consensus.cc:2258] T 307111a1c1db48a48ea2bdbe8201c64b P 4dbbedeffc64433bb1a5998b3dd93a32 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:34:55.691141 3088 raft_consensus.cc:2258] T 307111a1c1db48a48ea2bdbe8201c64b P 1c143d8a62f14f50bd97ed08f3698200 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:34:55.694610 3088 ts_tablet_manager.cc:1905] T 307111a1c1db48a48ea2bdbe8201c64b P 1c143d8a62f14f50bd97ed08f3698200: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:34:55.696058 3089 tablet_replica.cc:330] T 307111a1c1db48a48ea2bdbe8201c64b P de8a97824f5a4040b0a5cf456b22dfb6: stopping tablet replica
I20260806 06:34:55.696167 3089 raft_consensus.cc:2229] T 307111a1c1db48a48ea2bdbe8201c64b P de8a97824f5a4040b0a5cf456b22dfb6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:34:55.696259 3089 raft_consensus.cc:2258] T 307111a1c1db48a48ea2bdbe8201c64b P de8a97824f5a4040b0a5cf456b22dfb6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:34:55.696403 3088 ts_tablet_manager.cc:1918] T 307111a1c1db48a48ea2bdbe8201c64b P 1c143d8a62f14f50bd97ed08f3698200: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:34:55.696498 3088 log.cc:1196] T 307111a1c1db48a48ea2bdbe8201c64b P 1c143d8a62f14f50bd97ed08f3698200: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/ts-1/wal/wals/307111a1c1db48a48ea2bdbe8201c64b
I20260806 06:34:55.696758 3088 ts_tablet_manager.cc:1939] T 307111a1c1db48a48ea2bdbe8201c64b P 1c143d8a62f14f50bd97ed08f3698200: Deleting consensus metadata
I20260806 06:34:55.697155 2337 catalog_manager.cc:4841] TS 1c143d8a62f14f50bd97ed08f3698200 (127.7.43.66:41427): tablet 307111a1c1db48a48ea2bdbe8201c64b (table test-table [id=383c086b7a4644989c794c8961e7d201]) successfully deleted
I20260806 06:34:55.697821 3089 ts_tablet_manager.cc:1905] T 307111a1c1db48a48ea2bdbe8201c64b P de8a97824f5a4040b0a5cf456b22dfb6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:34:55.699062 3089 ts_tablet_manager.cc:1918] T 307111a1c1db48a48ea2bdbe8201c64b P de8a97824f5a4040b0a5cf456b22dfb6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:34:55.699119 3089 log.cc:1196] T 307111a1c1db48a48ea2bdbe8201c64b P de8a97824f5a4040b0a5cf456b22dfb6: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785997861828271-7341-0/minicluster-data/ts-2/wal/wals/307111a1c1db48a48ea2bdbe8201c64b
I20260806 06:34:55.699312 3089 ts_tablet_manager.cc:1939] T 307111a1c1db48a48ea2bdbe8201c64b P de8a97824f5a4040b0a5cf456b22dfb6: Deleting consensus metadata
I20260806 06:34:55.699715 2337 catalog_manager.cc:4841] TS de8a97824f5a4040b0a5cf456b22dfb6 (127.7.43.67:42789): tablet 307111a1c1db48a48ea2bdbe8201c64b (table test-table [id=383c086b7a4644989c794c8961e7d201]) successfully deleted
W20260806 06:34:55.702690 2337 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.43.65:46207 (error 108)
I20260806 06:34:55.702917 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 2644
I20260806 06:34:55.703099 3053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:55.702996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:46207 (local address 127.0.0.1:40316)
0806 06:34:55.703034 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:55.703069 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.65:46207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260806 06:34:55.706696 2337 catalog_manager.cc:4584] TS 4dbbedeffc64433bb1a5998b3dd93a32 (127.7.43.65:46207): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 307111a1c1db48a48ea2bdbe8201c64b: Network error: Client connection negotiation failed: client connection to 127.7.43.65:46207: connect: Connection refused (error 111)
I20260806 06:34:55.714942 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 2841
I20260806 06:34:55.723582 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 2310
2026-08-06T06:34:55Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (800 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:34:55 dist-test-slave-6zbq krb5kdc[3093](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:34:55 dist-test-slave-6zbq krb5kdc[3093](info): set up 2 sockets
Aug 06 06:34:55 dist-test-slave-6zbq krb5kdc[3093](info): commencing operation
krb5kdc: starting...
W20260806 06:34:56.832276 7341 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Aug 06 06:34:56 dist-test-slave-6zbq krb5kdc[3093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998096, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:34:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:34:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.43.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
I20260806 06:34:56.934875 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:38633
--webserver_interface=127.7.43.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:40267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.43.126:38633
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:34:57.012871 3108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:34:57.013044 3108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:34:57.013064 3108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:34:57.014146 3108 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:34:57.014185 3108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:34:57.014197 3108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:34:57.014220 3108 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:34:57.014243 3108 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:34:57.015682 3108 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:40267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.43.126:38633
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:38633
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.43.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3108
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:34:57.015901 3108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:34:57.016090 3108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:34:57.020522 3115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:34:57.020566 3108 server_base.cc:1034] running on GCE node
W20260806 06:34:57.020542 3116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:34:57.020514 3118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:34:57.020910 3108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:34:57.021174 3108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:34:57.022315 3108 hybrid_clock.cc:648] HybridClock initialized: now 1785998097022295 us; error 32 us; skew 500 ppm
Aug 06 06:34:57 dist-test-slave-6zbq krb5kdc[3093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998097, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:34:57.024544 3108 init.cc:377] Logged in from keytab as kudu/127.7.43.126@KRBTEST.COM (short username kudu)
I20260806 06:34:57.024945 3108 webserver.cc:460] Webserver started at http://127.7.43.126:38435/ using document root <none> and password file <none>
I20260806 06:34:57.025123 3108 fs_manager.cc:359] Metadata directory not provided
I20260806 06:34:57.025168 3108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:34:57.025286 3108 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:34:57.026213 3108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/master-0/data/instance:
uuid: "91393d243a9b42f0b5cc68727e5ac521"
format_stamp: "Formatted at 2026-08-06 06:34:57 on dist-test-slave-6zbq"
server_key: "de2a12f83c04857c1bf8e78df8797d75"
server_key_iv: "564df9e452efccad09a6a3fa0dc15f85"
server_key_version: "encryptionkey@0"
I20260806 06:34:57.026547 3108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/master-0/wal/instance:
uuid: "91393d243a9b42f0b5cc68727e5ac521"
format_stamp: "Formatted at 2026-08-06 06:34:57 on dist-test-slave-6zbq"
server_key: "de2a12f83c04857c1bf8e78df8797d75"
server_key_iv: "564df9e452efccad09a6a3fa0dc15f85"
server_key_version: "encryptionkey@0"
I20260806 06:34:57.027699 3108 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:34:57.028323 3126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:57.028466 3108 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:34:57.028543 3108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "91393d243a9b42f0b5cc68727e5ac521"
format_stamp: "Formatted at 2026-08-06 06:34:57 on dist-test-slave-6zbq"
server_key: "de2a12f83c04857c1bf8e78df8797d75"
server_key_iv: "564df9e452efccad09a6a3fa0dc15f85"
server_key_version: "encryptionkey@0"
I20260806 06:34:57.028594 3108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:34:57.044329 3108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:34:57.046190 3108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:34:57.046288 3108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:34:57.049878 3108 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.126:38633
I20260806 06:34:57.049909 3208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.126:38633 every 8 connection(s)
I20260806 06:34:57.050181 3108 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:34:57.050947 3209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:34:57.052964 3209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91393d243a9b42f0b5cc68727e5ac521: Bootstrap starting.
I20260806 06:34:57.053484 3209 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91393d243a9b42f0b5cc68727e5ac521: Neither blocks nor log segments found. Creating new log.
I20260806 06:34:57.053743 3209 log.cc:824] T 00000000000000000000000000000000 P 91393d243a9b42f0b5cc68727e5ac521: Log is configured to *not* fsync() on all Append() calls
I20260806 06:34:57.054332 3209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91393d243a9b42f0b5cc68727e5ac521: No bootstrap required, opened a new log
I20260806 06:34:57.055858 3209 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91393d243a9b42f0b5cc68727e5ac521 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91393d243a9b42f0b5cc68727e5ac521" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 38633 } }
I20260806 06:34:57.055984 3209 raft_consensus.cc:374] T 00000000000000000000000000000000 P 91393d243a9b42f0b5cc68727e5ac521 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:34:57.056015 3209 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91393d243a9b42f0b5cc68727e5ac521 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91393d243a9b42f0b5cc68727e5ac521, State: Initialized, Role: FOLLOWER
I20260806 06:34:57.056144 3209 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91393d243a9b42f0b5cc68727e5ac521 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91393d243a9b42f0b5cc68727e5ac521" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 38633 } }
I20260806 06:34:57.056202 3209 raft_consensus.cc:388] T 00000000000000000000000000000000 P 91393d243a9b42f0b5cc68727e5ac521 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:34:57.056234 3209 raft_consensus.cc:482] T 00000000000000000000000000000000 P 91393d243a9b42f0b5cc68727e5ac521 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:34:57.056278 3209 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91393d243a9b42f0b5cc68727e5ac521 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:34:57.056764 3209 raft_consensus.cc:504] T 00000000000000000000000000000000 P 91393d243a9b42f0b5cc68727e5ac521 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91393d243a9b42f0b5cc68727e5ac521" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 38633 } }
I20260806 06:34:57.056855 3209 leader_election.cc:302] T 00000000000000000000000000000000 P 91393d243a9b42f0b5cc68727e5ac521 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 91393d243a9b42f0b5cc68727e5ac521; no voters:
I20260806 06:34:57.057005 3209 leader_election.cc:288] T 00000000000000000000000000000000 P 91393d243a9b42f0b5cc68727e5ac521 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:34:57.057035 3213 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 91393d243a9b42f0b5cc68727e5ac521 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:34:57.057154 3213 raft_consensus.cc:686] T 00000000000000000000000000000000 P 91393d243a9b42f0b5cc68727e5ac521 [term 1 LEADER]: Becoming Leader. State: Replica: 91393d243a9b42f0b5cc68727e5ac521, State: Running, Role: LEADER
I20260806 06:34:57.057235 3209 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91393d243a9b42f0b5cc68727e5ac521 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:34:57.057296 3213 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91393d243a9b42f0b5cc68727e5ac521 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91393d243a9b42f0b5cc68727e5ac521" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 38633 } }
I20260806 06:34:57.057626 3219 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91393d243a9b42f0b5cc68727e5ac521 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91393d243a9b42f0b5cc68727e5ac521. Latest consensus state: current_term: 1 leader_uuid: "91393d243a9b42f0b5cc68727e5ac521" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91393d243a9b42f0b5cc68727e5ac521" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 38633 } } }
I20260806 06:34:57.057729 3214 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91393d243a9b42f0b5cc68727e5ac521 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "91393d243a9b42f0b5cc68727e5ac521" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91393d243a9b42f0b5cc68727e5ac521" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 38633 } } }
I20260806 06:34:57.057775 3219 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91393d243a9b42f0b5cc68727e5ac521 [sys.catalog]: This master's current role is: LEADER
I20260806 06:34:57.058156 3214 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91393d243a9b42f0b5cc68727e5ac521 [sys.catalog]: This master's current role is: LEADER
I20260806 06:34:57.058199 3230 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:34:57.058692 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 3108
I20260806 06:34:57.058751 3230 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:34:57.058827 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/master-0/wal/instance
I20260806 06:34:57.059012 7341 external_mini_cluster.cc:1428] Setting key f40038d2162eaf5631d2cda7d253575f
I20260806 06:34:57.060160 3230 catalog_manager.cc:1347] Generated new cluster ID: 4fc3609bdb6447c6b232acf4eb317b51
I20260806 06:34:57.060212 3230 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:34:57 dist-test-slave-6zbq krb5kdc[3093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998096, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:34:57.067943 3242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:57.059691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43536 (local address 127.7.43.126:38633)
0806 06:34:57.059837 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:57.059846 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:57.059877 (+ 31us) server_negotiation.cc:407] Connection header received
0806 06:34:57.059969 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:57.059974 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:57.060013 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:57.060126 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:34:57.060839 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:57.061735 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:57.062512 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:57.062933 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:57.063024 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:57.065056 (+ 2032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:34:57.065060 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:34:57.065060 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:34:57.065072 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:34:57.066562 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:34:57.066887 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:34:57.066891 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:34:57.066891 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:34:57.066987 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:34:57.067169 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:34:57.067170 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:34:57.067170 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:34:57.067385 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:34:57.067523 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:57.067633 (+ 110us) server_negotiation.cc:299] Negotiation successful
0806 06:34:57.067766 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":72,"threads_started":1}
I20260806 06:34:57.074146 3230 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:34:57.074826 3230 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:34:57.078871 3230 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 91393d243a9b42f0b5cc68727e5ac521: Generated new TSK 0
I20260806 06:34:57.079046 3230 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.43.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab.
Entry for principal kudu/127.7.43.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab.
I20260806 06:34:57.109719 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.65:0
--local_ip_for_outbound_sockets=127.7.43.65
--webserver_interface=127.7.43.65
--webserver_port=0
--tserver_master_addrs=127.7.43.126:38633
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab
--principal=kudu/127.7.43.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:40267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:34:57.187671 3248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:34:57.187857 3248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:34:57.187878 3248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:34:57.188958 3248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:34:57.188997 3248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:34:57.189025 3248 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.65
I20260806 06:34:57.190440 3248 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:40267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.65.keytab
--principal=kudu/127.7.43.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.43.65
--webserver_port=0
--tserver_master_addrs=127.7.43.126:38633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3248
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.65
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:34:57.190647 3248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:34:57.190845 3248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:34:57.195289 3257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:34:57.195386 3248 server_base.cc:1034] running on GCE node
W20260806 06:34:57.195290 3254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:34:57.195329 3255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:34:57.195680 3248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:34:57.195899 3248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:34:57.197041 3248 hybrid_clock.cc:648] HybridClock initialized: now 1785998097197026 us; error 29 us; skew 500 ppm
Aug 06 06:34:57 dist-test-slave-6zbq krb5kdc[3093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998097, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:34:57.199306 3248 init.cc:377] Logged in from keytab as kudu/127.7.43.65@KRBTEST.COM (short username kudu)
I20260806 06:34:57.199674 3248 webserver.cc:460] Webserver started at http://127.7.43.65:40469/ using document root <none> and password file <none>
I20260806 06:34:57.199875 3248 fs_manager.cc:359] Metadata directory not provided
I20260806 06:34:57.199920 3248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:34:57.200035 3248 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:34:57.200946 3248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/ts-0/data/instance:
uuid: "887a058ef15146e8a7f1b4f17d27639f"
format_stamp: "Formatted at 2026-08-06 06:34:57 on dist-test-slave-6zbq"
server_key: "9a2d5152a54f3276f22afc6f17374e44"
server_key_iv: "cc65d6f266ca8b35f5177c6e834b8771"
server_key_version: "encryptionkey@0"
I20260806 06:34:57.201270 3248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/ts-0/wal/instance:
uuid: "887a058ef15146e8a7f1b4f17d27639f"
format_stamp: "Formatted at 2026-08-06 06:34:57 on dist-test-slave-6zbq"
server_key: "9a2d5152a54f3276f22afc6f17374e44"
server_key_iv: "cc65d6f266ca8b35f5177c6e834b8771"
server_key_version: "encryptionkey@0"
I20260806 06:34:57.202471 3248 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:34:57.203104 3268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:57.203255 3248 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:57.203344 3248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/ts-0/wal
uuid: "887a058ef15146e8a7f1b4f17d27639f"
format_stamp: "Formatted at 2026-08-06 06:34:57 on dist-test-slave-6zbq"
server_key: "9a2d5152a54f3276f22afc6f17374e44"
server_key_iv: "cc65d6f266ca8b35f5177c6e834b8771"
server_key_version: "encryptionkey@0"
I20260806 06:34:57.203409 3248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:34:57.214447 3248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:34:57.216044 3248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:34:57.216169 3248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:34:57.216352 3248 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:34:57.216609 3248 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:34:57.216641 3248 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:57.216666 3248 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:34:57.216691 3248 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:57.221983 3248 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.65:38863
I20260806 06:34:57.222055 3438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.65:38863 every 8 connection(s)
I20260806 06:34:57.222295 3248 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
I20260806 06:34:57.223994 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 3248
I20260806 06:34:57.224100 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/ts-0/wal/instance
I20260806 06:34:57.224277 7341 external_mini_cluster.cc:1428] Setting key b0077b788f65185cd800d6453d1d646e
Aug 06 06:34:57 dist-test-slave-6zbq krb5kdc[3093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998097, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.65@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:34:57.231081 3242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:57.222935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:57633 (local address 127.7.43.126:38633)
0806 06:34:57.222976 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:57.222978 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:57.223142 (+ 164us) server_negotiation.cc:407] Connection header received
0806 06:34:57.223596 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:57.223597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:57.223612 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:57.223663 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:34:57.224669 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:57.225275 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:57.226219 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:57.226473 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:57.226524 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:57.228423 (+ 1899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:34:57.228424 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:34:57.228425 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:34:57.228433 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:34:57.229824 (+ 1391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:34:57.230172 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:34:57.230173 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:34:57.230173 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:34:57.230234 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:34:57.230501 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:34:57.230502 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:34:57.230503 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:34:57.230635 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:34:57.230707 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:57.230914 (+ 207us) server_negotiation.cc:299] Negotiation successful
0806 06:34:57.230993 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:34:57.231251 3441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:57.222887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38633 (local address 127.7.43.65:57633)
0806 06:34:57.223036 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:57.223069 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:57.223504 (+ 435us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:57.223727 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:57.223731 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:57.224323 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:34:57.224577 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:57.224585 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:57.225367 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:57.225368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:57.226138 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:57.226141 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:57.226551 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:57.226552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:57.226690 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:57.227097 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:34:57.227105 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:34:57.228337 (+ 1232us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:34:57.229901 (+ 1564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:34:57.229907 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:34:57.229909 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:34:57.230105 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:34:57.230295 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:34:57.230295 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:34:57.230295 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:34:57.230441 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:34:57.230695 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:34:57.230698 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:34:57.230799 (+ 101us) client_negotiation.cc:769] Sending connection context
0806 06:34:57.230871 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 06:34:57.231059 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":75,"threads_started":1}
I20260806 06:34:57.231689 3439 heartbeater.cc:347] Connected to a master server at 127.7.43.126:38633
I20260806 06:34:57.231792 3439 heartbeater.cc:464] Registering TS with master...
I20260806 06:34:57.231971 3439 heartbeater.cc:511] Master 127.7.43.126:38633 requested a full tablet report, sending...
I20260806 06:34:57.232462 3147 ts_manager.cc:194] Registered new tserver with Master: 887a058ef15146e8a7f1b4f17d27639f (127.7.43.65:38863)
I20260806 06:34:57.233013 3147 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.65@KRBTEST.COM'} at 127.7.43.65:57633
WARNING: no policy specified for kudu/127.7.43.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab.
Entry for principal kudu/127.7.43.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab.
I20260806 06:34:57.251535 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.66:0
--local_ip_for_outbound_sockets=127.7.43.66
--webserver_interface=127.7.43.66
--webserver_port=0
--tserver_master_addrs=127.7.43.126:38633
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab
--principal=kudu/127.7.43.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:40267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:34:57.332139 3448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:34:57.332319 3448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:34:57.332347 3448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:34:57.333432 3448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:34:57.333472 3448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:34:57.333504 3448 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.66
I20260806 06:34:57.334941 3448 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:40267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.66.keytab
--principal=kudu/127.7.43.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.43.66
--webserver_port=0
--tserver_master_addrs=127.7.43.126:38633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3448
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.66
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:34:57.335165 3448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:34:57.335364 3448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:34:57.340077 3458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:34:57.340106 3463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:34:57.340222 3448 server_base.cc:1034] running on GCE node
W20260806 06:34:57.340078 3460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:34:57.340493 3448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:34:57.340705 3448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:34:57.341856 3448 hybrid_clock.cc:648] HybridClock initialized: now 1785998097341843 us; error 34 us; skew 500 ppm
Aug 06 06:34:57 dist-test-slave-6zbq krb5kdc[3093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998097, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:34:57.344175 3448 init.cc:377] Logged in from keytab as kudu/127.7.43.66@KRBTEST.COM (short username kudu)
I20260806 06:34:57.344549 3448 webserver.cc:460] Webserver started at http://127.7.43.66:43911/ using document root <none> and password file <none>
I20260806 06:34:57.344763 3448 fs_manager.cc:359] Metadata directory not provided
I20260806 06:34:57.344810 3448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:34:57.344933 3448 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:34:57.345901 3448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/ts-1/data/instance:
uuid: "69ea3ccbbffa4aec932287975c1e5a9f"
format_stamp: "Formatted at 2026-08-06 06:34:57 on dist-test-slave-6zbq"
server_key: "98f93496cc9884d41410c80386a1d96c"
server_key_iv: "8464ef66c65430d058b19c4a1666a010"
server_key_version: "encryptionkey@0"
I20260806 06:34:57.346213 3448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/ts-1/wal/instance:
uuid: "69ea3ccbbffa4aec932287975c1e5a9f"
format_stamp: "Formatted at 2026-08-06 06:34:57 on dist-test-slave-6zbq"
server_key: "98f93496cc9884d41410c80386a1d96c"
server_key_iv: "8464ef66c65430d058b19c4a1666a010"
server_key_version: "encryptionkey@0"
I20260806 06:34:57.347415 3448 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:34:57.348080 3474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:57.348230 3448 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:34:57.348311 3448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/ts-1/wal
uuid: "69ea3ccbbffa4aec932287975c1e5a9f"
format_stamp: "Formatted at 2026-08-06 06:34:57 on dist-test-slave-6zbq"
server_key: "98f93496cc9884d41410c80386a1d96c"
server_key_iv: "8464ef66c65430d058b19c4a1666a010"
server_key_version: "encryptionkey@0"
I20260806 06:34:57.348368 3448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:34:57.358688 3448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:34:57.360265 3448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:34:57.360386 3448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:34:57.360565 3448 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:34:57.360831 3448 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:34:57.360862 3448 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:57.360888 3448 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:34:57.360905 3448 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:57.366108 3448 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.66:43919
I20260806 06:34:57.366173 3654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.66:43919 every 8 connection(s)
I20260806 06:34:57.366454 3448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
I20260806 06:34:57.367050 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 3448
I20260806 06:34:57.367173 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/ts-1/wal/instance
I20260806 06:34:57.367350 7341 external_mini_cluster.cc:1428] Setting key b2d31ebce6b2aefe3e3ae229ac8bf346
Aug 06 06:34:57 dist-test-slave-6zbq krb5kdc[3093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998097, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.66@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:34:57.375746 3242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:57.367056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:37643 (local address 127.7.43.126:38633)
0806 06:34:57.367124 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:57.367127 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:57.367372 (+ 245us) server_negotiation.cc:407] Connection header received
0806 06:34:57.367915 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:57.367916 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:57.367927 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:57.367988 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:34:57.369093 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:57.369575 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:57.370654 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:57.370953 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:57.371014 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:57.373253 (+ 2239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:34:57.373254 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:34:57.373254 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:34:57.373261 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:34:57.374473 (+ 1212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:34:57.374911 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:34:57.374911 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:34:57.374912 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:34:57.374956 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:34:57.375192 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:34:57.375193 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:34:57.375193 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:34:57.375283 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:34:57.375327 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:57.375621 (+ 294us) server_negotiation.cc:299] Negotiation successful
0806 06:34:57.375685 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 06:34:57.376042 3658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:57.367005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38633 (local address 127.7.43.66:37643)
0806 06:34:57.367170 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:57.367265 (+ 95us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:57.367820 (+ 555us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:57.368053 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:57.368057 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:57.368740 (+ 683us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:34:57.368999 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:57.369006 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:57.369653 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:57.369654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:57.370579 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:57.370582 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:57.371038 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:57.371039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:57.371179 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:57.371618 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:34:57.371627 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:34:57.373162 (+ 1535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:34:57.374541 (+ 1379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:34:57.374548 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:34:57.374550 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:34:57.374834 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:34:57.375010 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:34:57.375011 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:34:57.375012 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:34:57.375134 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:34:57.375339 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:34:57.375345 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:34:57.375498 (+ 153us) client_negotiation.cc:769] Sending connection context
0806 06:34:57.375615 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 06:34:57.375771 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":73,"threads_started":1}
WARNING: no policy specified for kudu/127.7.43.67@KRBTEST.COM; defaulting to no policy
I20260806 06:34:57.376498 3655 heartbeater.cc:347] Connected to a master server at 127.7.43.126:38633
I20260806 06:34:57.376583 3655 heartbeater.cc:464] Registering TS with master...
I20260806 06:34:57.376735 3655 heartbeater.cc:511] Master 127.7.43.126:38633 requested a full tablet report, sending...
I20260806 06:34:57.377089 3147 ts_manager.cc:194] Registered new tserver with Master: 69ea3ccbbffa4aec932287975c1e5a9f (127.7.43.66:43919)
I20260806 06:34:57.377446 3147 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.66@KRBTEST.COM'} at 127.7.43.66:37643
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab.
Entry for principal kudu/127.7.43.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab.
I20260806 06:34:57.390564 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.67:0
--local_ip_for_outbound_sockets=127.7.43.67
--webserver_interface=127.7.43.67
--webserver_port=0
--tserver_master_addrs=127.7.43.126:38633
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab
--principal=kudu/127.7.43.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:40267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:34:57.468544 3662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:34:57.468740 3662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:34:57.468771 3662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:34:57.469868 3662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:34:57.469908 3662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:34:57.469942 3662 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.67
I20260806 06:34:57.471369 3662 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:40267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.67.keytab
--principal=kudu/127.7.43.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.43.67
--webserver_port=0
--tserver_master_addrs=127.7.43.126:38633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3662
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.67
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:34:57.471601 3662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:34:57.471796 3662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:34:57.476395 3668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:34:57.476437 3662 server_base.cc:1034] running on GCE node
W20260806 06:34:57.476406 3669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:34:57.476413 3672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:34:57.476817 3662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:34:57.477015 3662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:34:57.478168 3662 hybrid_clock.cc:648] HybridClock initialized: now 1785998097478147 us; error 35 us; skew 500 ppm
Aug 06 06:34:57 dist-test-slave-6zbq krb5kdc[3093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998097, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:34:57.480470 3662 init.cc:377] Logged in from keytab as kudu/127.7.43.67@KRBTEST.COM (short username kudu)
I20260806 06:34:57.480820 3662 webserver.cc:460] Webserver started at http://127.7.43.67:34613/ using document root <none> and password file <none>
I20260806 06:34:57.480988 3662 fs_manager.cc:359] Metadata directory not provided
I20260806 06:34:57.481026 3662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:34:57.481124 3662 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:34:57.482031 3662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/ts-2/data/instance:
uuid: "c213938d7b494bb08b57628b9f532dd7"
format_stamp: "Formatted at 2026-08-06 06:34:57 on dist-test-slave-6zbq"
server_key: "ccd8d732f69257024d06753c9a0d5d5c"
server_key_iv: "ffb58297a3abbe2757c350074df348d1"
server_key_version: "encryptionkey@0"
I20260806 06:34:57.482316 3662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/ts-2/wal/instance:
uuid: "c213938d7b494bb08b57628b9f532dd7"
format_stamp: "Formatted at 2026-08-06 06:34:57 on dist-test-slave-6zbq"
server_key: "ccd8d732f69257024d06753c9a0d5d5c"
server_key_iv: "ffb58297a3abbe2757c350074df348d1"
server_key_version: "encryptionkey@0"
I20260806 06:34:57.483476 3662 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:34:57.484051 3682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:57.484216 3662 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:57.484288 3662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/ts-2/wal
uuid: "c213938d7b494bb08b57628b9f532dd7"
format_stamp: "Formatted at 2026-08-06 06:34:57 on dist-test-slave-6zbq"
server_key: "ccd8d732f69257024d06753c9a0d5d5c"
server_key_iv: "ffb58297a3abbe2757c350074df348d1"
server_key_version: "encryptionkey@0"
I20260806 06:34:57.484336 3662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:34:57.504499 3662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:34:57.506484 3662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:34:57.506585 3662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:34:57.506753 3662 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:34:57.506982 3662 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:34:57.507010 3662 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:57.507098 3662 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:34:57.507124 3662 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:57.512483 3662 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.67:38105
I20260806 06:34:57.512519 3858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.67:38105 every 8 connection(s)
I20260806 06:34:57.512787 3662 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
I20260806 06:34:57.514540 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 3662
I20260806 06:34:57.514631 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/ts-2/wal/instance
I20260806 06:34:57.514787 7341 external_mini_cluster.cc:1428] Setting key e6f2fd18dcb87d28672c5f16b0277776
Aug 06 06:34:57 dist-test-slave-6zbq krb5kdc[3093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998097, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.67@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:34:57.520684 3242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:57.513383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:33153 (local address 127.7.43.126:38633)
0806 06:34:57.513441 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:57.513443 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:57.513594 (+ 151us) server_negotiation.cc:407] Connection header received
0806 06:34:57.514068 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:57.514069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:57.514081 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:57.514139 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:34:57.515081 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:57.515558 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:57.516483 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:57.516671 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:57.516724 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:57.518524 (+ 1800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:34:57.518526 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:34:57.518526 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:34:57.518533 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:34:57.519515 (+ 982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:34:57.519855 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:34:57.519856 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:34:57.519856 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:34:57.519909 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:34:57.520112 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:34:57.520113 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:34:57.520113 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:34:57.520220 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:34:57.520276 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:57.520519 (+ 243us) server_negotiation.cc:299] Negotiation successful
0806 06:34:57.520598 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 06:34:57.520815 3862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:57.513350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38633 (local address 127.7.43.67:33153)
0806 06:34:57.513486 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:57.513519 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:57.513971 (+ 452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:57.514205 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:57.514208 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:57.514793 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:34:57.514991 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:57.514996 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:57.515638 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:57.515639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:57.516400 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:57.516402 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:57.516737 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:57.516738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:57.516857 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:57.517195 (+ 338us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:34:57.517202 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:34:57.518443 (+ 1241us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:34:57.519584 (+ 1141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:34:57.519590 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:34:57.519592 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:34:57.519790 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:34:57.519962 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:34:57.519963 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:34:57.519963 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:34:57.520058 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:34:57.520278 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:34:57.520285 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:34:57.520406 (+ 121us) client_negotiation.cc:769] Sending connection context
0806 06:34:57.520488 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 06:34:57.520627 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":69,"threads_started":1}
I20260806 06:34:57.521215 3860 heartbeater.cc:347] Connected to a master server at 127.7.43.126:38633
I20260806 06:34:57.521301 3860 heartbeater.cc:464] Registering TS with master...
I20260806 06:34:57.521426 3860 heartbeater.cc:511] Master 127.7.43.126:38633 requested a full tablet report, sending...
I20260806 06:34:57.521746 3147 ts_manager.cc:194] Registered new tserver with Master: c213938d7b494bb08b57628b9f532dd7 (127.7.43.67:38105)
I20260806 06:34:57.522074 3147 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.43.67@KRBTEST.COM'} at 127.7.43.67:33153
I20260806 06:34:57.526185 7341 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:34:57.532445 3242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:57.527434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43538 (local address 127.7.43.126:38633)
0806 06:34:57.527495 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:57.527496 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:57.527576 (+ 80us) server_negotiation.cc:407] Connection header received
0806 06:34:57.527616 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:57.527616 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:57.527630 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:57.527672 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:34:57.528135 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:57.528556 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:57.529217 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:57.529389 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:57.529430 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:57.530376 (+ 946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:34:57.530378 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:34:57.530378 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:34:57.530385 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:34:57.531607 (+ 1222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:34:57.531868 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:34:57.531869 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:34:57.531869 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:34:57.531913 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:34:57.532028 (+ 115us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:34:57.532028 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:34:57.532028 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:34:57.532154 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:34:57.532221 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:57.532327 (+ 106us) server_negotiation.cc:299] Negotiation successful
0806 06:34:57.532382 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 06:34:57.533721 3147 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43538:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:34:57.535259 3147 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:34:57.540999 3874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:57.537129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.67:38105 (local address 127.0.0.1:46084)
0806 06:34:57.537583 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:57.537594 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:57.537658 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:57.537851 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:57.537854 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:57.537857 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:34:57.537939 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:57.537941 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:57.538898 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:57.538899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:57.539941 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:57.539944 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:57.540729 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:57.540730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:57.540809 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:57.540822 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 06:34:57.540882 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 06:34:57.540931 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":61,"threads_started":1}
I20260806 06:34:57.541005 3875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:57.537411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46084 (local address 127.7.43.67:38105)
0806 06:34:57.537565 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:57.537570 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:57.537645 (+ 75us) server_negotiation.cc:407] Connection header received
0806 06:34:57.537729 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:57.537731 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:57.537748 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:57.537827 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:34:57.538017 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:57.538809 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:57.540033 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:57.540658 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:57.540714 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:57.540768 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:57.540901 (+ 133us) server_negotiation.cc:299] Negotiation successful
0806 06:34:57.540935 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":79,"threads_started":1}
I20260806 06:34:57.541388 3876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:57.537423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39776 (local address 127.7.43.66:43919)
0806 06:34:57.537576 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:57.537580 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:57.537851 (+ 271us) server_negotiation.cc:407] Connection header received
0806 06:34:57.537966 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:57.537968 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:57.537983 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:57.538036 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:34:57.538255 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:57.539313 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:57.540303 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:57.540932 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:57.540996 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:57.541044 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:57.541268 (+ 224us) server_negotiation.cc:299] Negotiation successful
0806 06:34:57.541309 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":72,"threads_started":1}
I20260806 06:34:57.541445 3871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:57.537125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:38863 (local address 127.0.0.1:36216)
0806 06:34:57.537221 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:57.537255 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:57.537313 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:57.537952 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:57.537953 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:57.537956 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:34:57.538002 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:57.538004 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:57.539341 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:57.539342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:57.540409 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:57.540412 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:57.541260 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:57.541261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:57.541315 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:57.541318 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:34:57.541346 (+ 28us) client_negotiation.cc:240] Negotiation successful
0806 06:34:57.541382 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73,"mutex_wait_us":22,"thread_start_us":47,"threads_started":1}
I20260806 06:34:57.541553 3872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:57.537293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.66:43919 (local address 127.0.0.1:39776)
0806 06:34:57.537788 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:57.537795 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:57.537877 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:57.538138 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:57.538139 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:57.538141 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:34:57.538182 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:57.538183 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:57.539389 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:57.539390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:57.540234 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:57.540236 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:57.541080 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:57.541081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:57.541156 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:57.541160 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:34:57.541466 (+ 306us) client_negotiation.cc:240] Negotiation successful
0806 06:34:57.541501 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"mutex_wait_us":1,"thread_start_us":62,"threads_started":1}
I20260806 06:34:57.541724 3877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:57.537591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36216 (local address 127.7.43.65:38863)
0806 06:34:57.537787 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:57.537791 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:57.537800 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:34:57.537838 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:57.537841 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:57.537854 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:57.538074 (+ 220us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:34:57.538153 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:57.539241 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:57.540489 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:57.541177 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:57.541531 (+ 354us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:57.541598 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:57.541623 (+ 25us) server_negotiation.cc:299] Negotiation successful
0806 06:34:57.541657 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":105,"threads_started":1}
I20260806 06:34:57.541970 3758 tablet_service.cc:1467] Processing CreateTablet for tablet 72a906fbc86b4686987943207c7081af (DEFAULT_TABLE table=test-table [id=7166a5d8906b4ac3ba3026e233a53844]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:34:57.542248 3758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72a906fbc86b4686987943207c7081af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:34:57.544512 3878 tablet_bootstrap.cc:492] T 72a906fbc86b4686987943207c7081af P c213938d7b494bb08b57628b9f532dd7: Bootstrap starting.
I20260806 06:34:57.545070 3347 tablet_service.cc:1467] Processing CreateTablet for tablet 72a906fbc86b4686987943207c7081af (DEFAULT_TABLE table=test-table [id=7166a5d8906b4ac3ba3026e233a53844]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:34:57.545203 3878 tablet_bootstrap.cc:654] T 72a906fbc86b4686987943207c7081af P c213938d7b494bb08b57628b9f532dd7: Neither blocks nor log segments found. Creating new log.
I20260806 06:34:57.545403 3347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72a906fbc86b4686987943207c7081af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:34:57.545303 3551 tablet_service.cc:1467] Processing CreateTablet for tablet 72a906fbc86b4686987943207c7081af (DEFAULT_TABLE table=test-table [id=7166a5d8906b4ac3ba3026e233a53844]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:34:57.545601 3551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72a906fbc86b4686987943207c7081af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:34:57.545742 3878 log.cc:824] T 72a906fbc86b4686987943207c7081af P c213938d7b494bb08b57628b9f532dd7: Log is configured to *not* fsync() on all Append() calls
I20260806 06:34:57.546380 3878 tablet_bootstrap.cc:492] T 72a906fbc86b4686987943207c7081af P c213938d7b494bb08b57628b9f532dd7: No bootstrap required, opened a new log
I20260806 06:34:57.546499 3878 ts_tablet_manager.cc:1397] T 72a906fbc86b4686987943207c7081af P c213938d7b494bb08b57628b9f532dd7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:34:57.547781 3878 raft_consensus.cc:348] T 72a906fbc86b4686987943207c7081af P c213938d7b494bb08b57628b9f532dd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "887a058ef15146e8a7f1b4f17d27639f" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 38863 } } peers { permanent_uuid: "c213938d7b494bb08b57628b9f532dd7" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 38105 } } peers { permanent_uuid: "69ea3ccbbffa4aec932287975c1e5a9f" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 43919 } }
I20260806 06:34:57.547909 3878 raft_consensus.cc:374] T 72a906fbc86b4686987943207c7081af P c213938d7b494bb08b57628b9f532dd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:34:57.547941 3878 raft_consensus.cc:729] T 72a906fbc86b4686987943207c7081af P c213938d7b494bb08b57628b9f532dd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c213938d7b494bb08b57628b9f532dd7, State: Initialized, Role: FOLLOWER
I20260806 06:34:57.548069 3878 consensus_queue.cc:260] T 72a906fbc86b4686987943207c7081af P c213938d7b494bb08b57628b9f532dd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "887a058ef15146e8a7f1b4f17d27639f" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 38863 } } peers { permanent_uuid: "c213938d7b494bb08b57628b9f532dd7" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 38105 } } peers { permanent_uuid: "69ea3ccbbffa4aec932287975c1e5a9f" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 43919 } }
I20260806 06:34:57.548333 3878 ts_tablet_manager.cc:1428] T 72a906fbc86b4686987943207c7081af P c213938d7b494bb08b57628b9f532dd7: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 06:34:57.548444 3860 heartbeater.cc:503] Master 127.7.43.126:38633 was elected leader, sending a full tablet report...
I20260806 06:34:57.548604 3888 tablet_bootstrap.cc:492] T 72a906fbc86b4686987943207c7081af P 887a058ef15146e8a7f1b4f17d27639f: Bootstrap starting.
I20260806 06:34:57.549029 3885 tablet_bootstrap.cc:492] T 72a906fbc86b4686987943207c7081af P 69ea3ccbbffa4aec932287975c1e5a9f: Bootstrap starting.
I20260806 06:34:57.549223 3888 tablet_bootstrap.cc:654] T 72a906fbc86b4686987943207c7081af P 887a058ef15146e8a7f1b4f17d27639f: Neither blocks nor log segments found. Creating new log.
I20260806 06:34:57.549719 3885 tablet_bootstrap.cc:654] T 72a906fbc86b4686987943207c7081af P 69ea3ccbbffa4aec932287975c1e5a9f: Neither blocks nor log segments found. Creating new log.
I20260806 06:34:57.549746 3888 log.cc:824] T 72a906fbc86b4686987943207c7081af P 887a058ef15146e8a7f1b4f17d27639f: Log is configured to *not* fsync() on all Append() calls
I20260806 06:34:57.550228 3885 log.cc:824] T 72a906fbc86b4686987943207c7081af P 69ea3ccbbffa4aec932287975c1e5a9f: Log is configured to *not* fsync() on all Append() calls
I20260806 06:34:57.550519 3888 tablet_bootstrap.cc:492] T 72a906fbc86b4686987943207c7081af P 887a058ef15146e8a7f1b4f17d27639f: No bootstrap required, opened a new log
I20260806 06:34:57.550575 3888 ts_tablet_manager.cc:1397] T 72a906fbc86b4686987943207c7081af P 887a058ef15146e8a7f1b4f17d27639f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:34:57.550937 3885 tablet_bootstrap.cc:492] T 72a906fbc86b4686987943207c7081af P 69ea3ccbbffa4aec932287975c1e5a9f: No bootstrap required, opened a new log
I20260806 06:34:57.551007 3885 ts_tablet_manager.cc:1397] T 72a906fbc86b4686987943207c7081af P 69ea3ccbbffa4aec932287975c1e5a9f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:34:57.552414 3888 raft_consensus.cc:348] T 72a906fbc86b4686987943207c7081af P 887a058ef15146e8a7f1b4f17d27639f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "887a058ef15146e8a7f1b4f17d27639f" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 38863 } } peers { permanent_uuid: "c213938d7b494bb08b57628b9f532dd7" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 38105 } } peers { permanent_uuid: "69ea3ccbbffa4aec932287975c1e5a9f" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 43919 } }
I20260806 06:34:57.552583 3888 raft_consensus.cc:374] T 72a906fbc86b4686987943207c7081af P 887a058ef15146e8a7f1b4f17d27639f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:34:57.552624 3888 raft_consensus.cc:729] T 72a906fbc86b4686987943207c7081af P 887a058ef15146e8a7f1b4f17d27639f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 887a058ef15146e8a7f1b4f17d27639f, State: Initialized, Role: FOLLOWER
I20260806 06:34:57.552733 3888 consensus_queue.cc:260] T 72a906fbc86b4686987943207c7081af P 887a058ef15146e8a7f1b4f17d27639f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "887a058ef15146e8a7f1b4f17d27639f" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 38863 } } peers { permanent_uuid: "c213938d7b494bb08b57628b9f532dd7" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 38105 } } peers { permanent_uuid: "69ea3ccbbffa4aec932287975c1e5a9f" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 43919 } }
I20260806 06:34:57.552809 3885 raft_consensus.cc:348] T 72a906fbc86b4686987943207c7081af P 69ea3ccbbffa4aec932287975c1e5a9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "887a058ef15146e8a7f1b4f17d27639f" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 38863 } } peers { permanent_uuid: "c213938d7b494bb08b57628b9f532dd7" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 38105 } } peers { permanent_uuid: "69ea3ccbbffa4aec932287975c1e5a9f" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 43919 } }
I20260806 06:34:57.552994 3888 ts_tablet_manager.cc:1428] T 72a906fbc86b4686987943207c7081af P 887a058ef15146e8a7f1b4f17d27639f: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 06:34:57.553004 3885 raft_consensus.cc:374] T 72a906fbc86b4686987943207c7081af P 69ea3ccbbffa4aec932287975c1e5a9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:34:57.552997 3439 heartbeater.cc:503] Master 127.7.43.126:38633 was elected leader, sending a full tablet report...
I20260806 06:34:57.553045 3885 raft_consensus.cc:729] T 72a906fbc86b4686987943207c7081af P 69ea3ccbbffa4aec932287975c1e5a9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69ea3ccbbffa4aec932287975c1e5a9f, State: Initialized, Role: FOLLOWER
I20260806 06:34:57.553171 3885 consensus_queue.cc:260] T 72a906fbc86b4686987943207c7081af P 69ea3ccbbffa4aec932287975c1e5a9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "887a058ef15146e8a7f1b4f17d27639f" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 38863 } } peers { permanent_uuid: "c213938d7b494bb08b57628b9f532dd7" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 38105 } } peers { permanent_uuid: "69ea3ccbbffa4aec932287975c1e5a9f" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 43919 } }
I20260806 06:34:57.553397 3885 ts_tablet_manager.cc:1428] T 72a906fbc86b4686987943207c7081af P 69ea3ccbbffa4aec932287975c1e5a9f: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 06:34:57.553426 3655 heartbeater.cc:503] Master 127.7.43.126:38633 was elected leader, sending a full tablet report...
W20260806 06:34:57.616968 3656 tablet.cc:2369] T 72a906fbc86b4686987943207c7081af P 69ea3ccbbffa4aec932287975c1e5a9f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:34:57.722236 3894 raft_consensus.cc:482] T 72a906fbc86b4686987943207c7081af P 887a058ef15146e8a7f1b4f17d27639f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:34:57.722383 3894 raft_consensus.cc:504] T 72a906fbc86b4686987943207c7081af P 887a058ef15146e8a7f1b4f17d27639f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "887a058ef15146e8a7f1b4f17d27639f" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 38863 } } peers { permanent_uuid: "c213938d7b494bb08b57628b9f532dd7" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 38105 } } peers { permanent_uuid: "69ea3ccbbffa4aec932287975c1e5a9f" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 43919 } }
I20260806 06:34:57.722735 3894 leader_election.cc:288] T 72a906fbc86b4686987943207c7081af P 887a058ef15146e8a7f1b4f17d27639f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c213938d7b494bb08b57628b9f532dd7 (127.7.43.67:38105), 69ea3ccbbffa4aec932287975c1e5a9f (127.7.43.66:43919)
W20260806 06:34:57.722924 3440 tablet.cc:2369] T 72a906fbc86b4686987943207c7081af P 887a058ef15146e8a7f1b4f17d27639f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:34:57.726374 3876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:57.722959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:35213 (local address 127.7.43.66:43919)
0806 06:34:57.722998 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:57.723000 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:57.723156 (+ 156us) server_negotiation.cc:407] Connection header received
0806 06:34:57.723275 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:57.723277 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:57.723290 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:57.723343 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:34:57.723459 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:57.724123 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:57.725435 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:57.726018 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:57.726089 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:57.726101 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:57.726252 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 06:34:57.726291 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:34:57.726413 3896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:57.722869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.66:43919 (local address 127.7.43.65:35213)
0806 06:34:57.723102 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:57.723110 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:57.723177 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:57.723357 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:57.723358 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:57.723359 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:34:57.723407 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:57.723408 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:57.724183 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:57.724184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:57.725350 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:57.725353 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:57.726093 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:57.726094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:57.726180 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:57.726184 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:34:57.726236 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 06:34:57.726278 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"mutex_wait_us":142,"thread_start_us":57,"threads_started":1}
I20260806 06:34:57.726579 3875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:57.722990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:56539 (local address 127.7.43.67:38105)
0806 06:34:57.723045 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:57.723047 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:57.723056 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:34:57.723082 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:57.723083 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:57.723094 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:57.723158 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:34:57.723328 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:57.724085 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:57.725435 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:57.726021 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:57.726089 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:57.726100 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:57.726477 (+ 377us) server_negotiation.cc:299] Negotiation successful
0806 06:34:57.726523 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 06:34:57.726419 3441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:57.722869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.67:38105 (local address 127.7.43.65:56539)
0806 06:34:57.722937 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:57.722947 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:57.722987 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:57.723172 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:57.723174 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:57.723176 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:34:57.723251 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:57.723254 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:57.724160 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:57.724161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:57.725350 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:57.725353 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:57.726093 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:57.726094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:57.726182 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:57.726184 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:34:57.726235 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 06:34:57.726278 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":30}
I20260806 06:34:57.726789 3583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72a906fbc86b4686987943207c7081af" candidate_uuid: "887a058ef15146e8a7f1b4f17d27639f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69ea3ccbbffa4aec932287975c1e5a9f" is_pre_election: true
I20260806 06:34:57.726924 3583 raft_consensus.cc:2454] T 72a906fbc86b4686987943207c7081af P 69ea3ccbbffa4aec932287975c1e5a9f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 887a058ef15146e8a7f1b4f17d27639f in term 0.
I20260806 06:34:57.726953 3790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72a906fbc86b4686987943207c7081af" candidate_uuid: "887a058ef15146e8a7f1b4f17d27639f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c213938d7b494bb08b57628b9f532dd7" is_pre_election: true
I20260806 06:34:57.727097 3790 raft_consensus.cc:2454] T 72a906fbc86b4686987943207c7081af P c213938d7b494bb08b57628b9f532dd7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 887a058ef15146e8a7f1b4f17d27639f in term 0.
I20260806 06:34:57.727144 3272 leader_election.cc:302] T 72a906fbc86b4686987943207c7081af P 887a058ef15146e8a7f1b4f17d27639f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69ea3ccbbffa4aec932287975c1e5a9f, 887a058ef15146e8a7f1b4f17d27639f; no voters:
I20260806 06:34:57.727275 3894 raft_consensus.cc:2781] T 72a906fbc86b4686987943207c7081af P 887a058ef15146e8a7f1b4f17d27639f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:34:57.727324 3894 raft_consensus.cc:482] T 72a906fbc86b4686987943207c7081af P 887a058ef15146e8a7f1b4f17d27639f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:34:57.727345 3894 raft_consensus.cc:3037] T 72a906fbc86b4686987943207c7081af P 887a058ef15146e8a7f1b4f17d27639f [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:34:57.728022 3894 raft_consensus.cc:504] T 72a906fbc86b4686987943207c7081af P 887a058ef15146e8a7f1b4f17d27639f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "887a058ef15146e8a7f1b4f17d27639f" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 38863 } } peers { permanent_uuid: "c213938d7b494bb08b57628b9f532dd7" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 38105 } } peers { permanent_uuid: "69ea3ccbbffa4aec932287975c1e5a9f" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 43919 } }
I20260806 06:34:57.728125 3894 leader_election.cc:288] T 72a906fbc86b4686987943207c7081af P 887a058ef15146e8a7f1b4f17d27639f [CANDIDATE]: Term 1 election: Requested vote from peers c213938d7b494bb08b57628b9f532dd7 (127.7.43.67:38105), 69ea3ccbbffa4aec932287975c1e5a9f (127.7.43.66:43919)
I20260806 06:34:57.728250 3790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72a906fbc86b4686987943207c7081af" candidate_uuid: "887a058ef15146e8a7f1b4f17d27639f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c213938d7b494bb08b57628b9f532dd7"
I20260806 06:34:57.728295 3583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72a906fbc86b4686987943207c7081af" candidate_uuid: "887a058ef15146e8a7f1b4f17d27639f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69ea3ccbbffa4aec932287975c1e5a9f"
I20260806 06:34:57.728327 3790 raft_consensus.cc:3037] T 72a906fbc86b4686987943207c7081af P c213938d7b494bb08b57628b9f532dd7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:34:57.728366 3583 raft_consensus.cc:3037] T 72a906fbc86b4686987943207c7081af P 69ea3ccbbffa4aec932287975c1e5a9f [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:34:57.729182 3583 raft_consensus.cc:2454] T 72a906fbc86b4686987943207c7081af P 69ea3ccbbffa4aec932287975c1e5a9f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 887a058ef15146e8a7f1b4f17d27639f in term 1.
I20260806 06:34:57.729188 3790 raft_consensus.cc:2454] T 72a906fbc86b4686987943207c7081af P c213938d7b494bb08b57628b9f532dd7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 887a058ef15146e8a7f1b4f17d27639f in term 1.
I20260806 06:34:57.729323 3272 leader_election.cc:302] T 72a906fbc86b4686987943207c7081af P 887a058ef15146e8a7f1b4f17d27639f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69ea3ccbbffa4aec932287975c1e5a9f, 887a058ef15146e8a7f1b4f17d27639f; no voters:
I20260806 06:34:57.729446 3894 raft_consensus.cc:2781] T 72a906fbc86b4686987943207c7081af P 887a058ef15146e8a7f1b4f17d27639f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:34:57.729594 3894 raft_consensus.cc:686] T 72a906fbc86b4686987943207c7081af P 887a058ef15146e8a7f1b4f17d27639f [term 1 LEADER]: Becoming Leader. State: Replica: 887a058ef15146e8a7f1b4f17d27639f, State: Running, Role: LEADER
I20260806 06:34:57.729686 3894 consensus_queue.cc:237] T 72a906fbc86b4686987943207c7081af P 887a058ef15146e8a7f1b4f17d27639f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "887a058ef15146e8a7f1b4f17d27639f" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 38863 } } peers { permanent_uuid: "c213938d7b494bb08b57628b9f532dd7" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 38105 } } peers { permanent_uuid: "69ea3ccbbffa4aec932287975c1e5a9f" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 43919 } }
I20260806 06:34:57.730432 3146 catalog_manager.cc:5495] T 72a906fbc86b4686987943207c7081af P 887a058ef15146e8a7f1b4f17d27639f reported cstate change: term changed from 0 to 1, leader changed from <none> to 887a058ef15146e8a7f1b4f17d27639f (127.7.43.65). New cstate: current_term: 1 leader_uuid: "887a058ef15146e8a7f1b4f17d27639f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "887a058ef15146e8a7f1b4f17d27639f" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 38863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c213938d7b494bb08b57628b9f532dd7" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 38105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "69ea3ccbbffa4aec932287975c1e5a9f" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 43919 } health_report { overall_health: UNKNOWN } } }
W20260806 06:34:57.763276 3861 tablet.cc:2369] T 72a906fbc86b4686987943207c7081af P c213938d7b494bb08b57628b9f532dd7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:34:57.767539 3877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:57.765410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36224 (local address 127.7.43.65:38863)
0806 06:34:57.765452 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:57.765454 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:57.765462 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:34:57.765513 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:57.765513 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:57.765531 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:57.765584 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:34:57.765707 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:57.766133 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:57.766871 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:57.767049 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:57.767100 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:57.767264 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:34:57.767351 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:34:57.767417 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:57.767455 (+ 38us) server_negotiation.cc:299] Negotiation successful
0806 06:34:57.767473 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:34:57.769045 3790 raft_consensus.cc:1261] T 72a906fbc86b4686987943207c7081af P c213938d7b494bb08b57628b9f532dd7 [term 1 FOLLOWER]: Refusing update from remote peer 887a058ef15146e8a7f1b4f17d27639f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:34:57.769045 3583 raft_consensus.cc:1261] T 72a906fbc86b4686987943207c7081af P 69ea3ccbbffa4aec932287975c1e5a9f [term 1 FOLLOWER]: Refusing update from remote peer 887a058ef15146e8a7f1b4f17d27639f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:34:57.769368 3897 consensus_queue.cc:1035] T 72a906fbc86b4686987943207c7081af P 887a058ef15146e8a7f1b4f17d27639f [LEADER]: Connected to new peer: Peer: permanent_uuid: "69ea3ccbbffa4aec932287975c1e5a9f" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 43919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:34:57.769486 3894 consensus_queue.cc:1035] T 72a906fbc86b4686987943207c7081af P 887a058ef15146e8a7f1b4f17d27639f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c213938d7b494bb08b57628b9f532dd7" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 38105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:34:57.772197 3901 mvcc.cc:205] Tried to move back new op lower bound from 7315448208459735040 to 7315448208301379584. Current Snapshot: MvccSnapshot[applied={T|T < 7315448208459735040 or (T in {7315448208459735040})}]
I20260806 06:34:57.773681 3146 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43538:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:34:57.773757 3146 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43538:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:34:57.774392 3146 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 91393d243a9b42f0b5cc68727e5ac521: Sending DeleteTablet for 3 replicas of tablet 72a906fbc86b4686987943207c7081af
I20260806 06:34:57.774672 3347 tablet_service.cc:1514] Processing DeleteTablet for tablet 72a906fbc86b4686987943207c7081af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:34:57 UTC) from {username='kudu', principal='kudu/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:36216
I20260806 06:34:57.774735 3551 tablet_service.cc:1514] Processing DeleteTablet for tablet 72a906fbc86b4686987943207c7081af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:34:57 UTC) from {username='kudu', principal='kudu/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:39776
I20260806 06:34:57.774783 3758 tablet_service.cc:1514] Processing DeleteTablet for tablet 72a906fbc86b4686987943207c7081af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:34:57 UTC) from {username='kudu', principal='kudu/127.7.43.126@KRBTEST.COM'} at 127.0.0.1:46084
I20260806 06:34:57.774900 3913 tablet_replica.cc:330] T 72a906fbc86b4686987943207c7081af P 887a058ef15146e8a7f1b4f17d27639f: stopping tablet replica
I20260806 06:34:57.774976 3915 tablet_replica.cc:330] T 72a906fbc86b4686987943207c7081af P 69ea3ccbbffa4aec932287975c1e5a9f: stopping tablet replica
I20260806 06:34:57.774989 3913 raft_consensus.cc:2229] T 72a906fbc86b4686987943207c7081af P 887a058ef15146e8a7f1b4f17d27639f [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:34:57.775063 3915 raft_consensus.cc:2229] T 72a906fbc86b4686987943207c7081af P 69ea3ccbbffa4aec932287975c1e5a9f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:34:57.775068 3913 raft_consensus.cc:2258] T 72a906fbc86b4686987943207c7081af P 887a058ef15146e8a7f1b4f17d27639f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:34:57.775148 3915 raft_consensus.cc:2258] T 72a906fbc86b4686987943207c7081af P 69ea3ccbbffa4aec932287975c1e5a9f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:34:57.775229 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 3248
I20260806 06:34:57.775363 3916 tablet_replica.cc:330] T 72a906fbc86b4686987943207c7081af P c213938d7b494bb08b57628b9f532dd7: stopping tablet replica
I20260806 06:34:57.775444 3916 raft_consensus.cc:2229] T 72a906fbc86b4686987943207c7081af P c213938d7b494bb08b57628b9f532dd7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:34:57.775525 3916 raft_consensus.cc:2258] T 72a906fbc86b4686987943207c7081af P c213938d7b494bb08b57628b9f532dd7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:34:57.783473 3915 ts_tablet_manager.cc:1905] T 72a906fbc86b4686987943207c7081af P 69ea3ccbbffa4aec932287975c1e5a9f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:34:57.785055 3916 ts_tablet_manager.cc:1905] T 72a906fbc86b4686987943207c7081af P c213938d7b494bb08b57628b9f532dd7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:34:57.785146 3915 ts_tablet_manager.cc:1918] T 72a906fbc86b4686987943207c7081af P 69ea3ccbbffa4aec932287975c1e5a9f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:34:57.785320 3915 log.cc:1196] T 72a906fbc86b4686987943207c7081af P 69ea3ccbbffa4aec932287975c1e5a9f: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/ts-1/wal/wals/72a906fbc86b4686987943207c7081af
I20260806 06:34:57.785717 3915 ts_tablet_manager.cc:1939] T 72a906fbc86b4686987943207c7081af P 69ea3ccbbffa4aec932287975c1e5a9f: Deleting consensus metadata
I20260806 06:34:57.786307 3916 ts_tablet_manager.cc:1918] T 72a906fbc86b4686987943207c7081af P c213938d7b494bb08b57628b9f532dd7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:34:57.786437 3916 log.cc:1196] T 72a906fbc86b4686987943207c7081af P c213938d7b494bb08b57628b9f532dd7: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785997861828271-7341-0/minicluster-data/ts-2/wal/wals/72a906fbc86b4686987943207c7081af
I20260806 06:34:57.786656 3130 catalog_manager.cc:4841] TS 69ea3ccbbffa4aec932287975c1e5a9f (127.7.43.66:43919): tablet 72a906fbc86b4686987943207c7081af (table test-table [id=7166a5d8906b4ac3ba3026e233a53844]) successfully deleted
I20260806 06:34:57.786780 3916 ts_tablet_manager.cc:1939] T 72a906fbc86b4686987943207c7081af P c213938d7b494bb08b57628b9f532dd7: Deleting consensus metadata
I20260806 06:34:57.787154 3132 catalog_manager.cc:4841] TS c213938d7b494bb08b57628b9f532dd7 (127.7.43.67:38105): tablet 72a906fbc86b4686987943207c7081af (table test-table [id=7166a5d8906b4ac3ba3026e233a53844]) successfully deleted
W20260806 06:34:57.791103 3130 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.43.65:38863 (error 108)
I20260806 06:34:57.791373 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 3448
I20260806 06:34:57.791504 3872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:57.791394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:38863 (local address 127.0.0.1:36230)
0806 06:34:57.791441 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:57.791478 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.65:38863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 06:34:57.792634 3130 catalog_manager.cc:4584] TS 887a058ef15146e8a7f1b4f17d27639f (127.7.43.65:38863): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 72a906fbc86b4686987943207c7081af: Network error: Client connection negotiation failed: client connection to 127.7.43.65:38863: connect: Connection refused (error 111)
I20260806 06:34:57.798962 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 3662
I20260806 06:34:57.807680 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 3108
2026-08-06T06:34:57Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2083 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-06T06:34:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:34:57Z Disabled control of system clock
I20260806 06:34:57.836200 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:38355
--webserver_interface=127.7.43.126
--webserver_port=0
--builtin_ntp_servers=127.7.43.84:42539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.43.126:38355
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 06:34:57.912935 3924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:34:57.913118 3924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:34:57.913136 3924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:34:57.914238 3924 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:34:57.914278 3924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:34:57.914290 3924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:34:57.914314 3924 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:34:57.914351 3924 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:34:57.915642 3924 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:42539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.43.126:38355
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:38355
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.43.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3924
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:34:57.915849 3924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:34:57.916093 3924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:34:57.920437 3932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:34:57.920416 3933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:34:57.920507 3924 server_base.cc:1034] running on GCE node
W20260806 06:34:57.920424 3936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:34:57.920856 3924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:34:57.921128 3924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:34:57.922271 3924 hybrid_clock.cc:648] HybridClock initialized: now 1785998097922257 us; error 27 us; skew 500 ppm
I20260806 06:34:57.923413 3924 webserver.cc:460] Webserver started at http://127.7.43.126:33647/ using document root <none> and password file <none>
I20260806 06:34:57.923614 3924 fs_manager.cc:359] Metadata directory not provided
I20260806 06:34:57.923660 3924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:34:57.923784 3924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:34:57.924719 3924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/master-0/data/instance:
uuid: "e2eb8807d5b14ca2861ccdf953c733f1"
format_stamp: "Formatted at 2026-08-06 06:34:57 on dist-test-slave-6zbq"
server_key: "20cbae5d31f9202be72f48be81018bc5"
server_key_iv: "ddafe310c8bf16fb730f2081bd6e671d"
server_key_version: "encryptionkey@0"
I20260806 06:34:57.925030 3924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/master-0/wal/instance:
uuid: "e2eb8807d5b14ca2861ccdf953c733f1"
format_stamp: "Formatted at 2026-08-06 06:34:57 on dist-test-slave-6zbq"
server_key: "20cbae5d31f9202be72f48be81018bc5"
server_key_iv: "ddafe310c8bf16fb730f2081bd6e671d"
server_key_version: "encryptionkey@0"
I20260806 06:34:57.926147 3924 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:34:57.926860 3944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:57.927006 3924 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:57.927084 3924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "e2eb8807d5b14ca2861ccdf953c733f1"
format_stamp: "Formatted at 2026-08-06 06:34:57 on dist-test-slave-6zbq"
server_key: "20cbae5d31f9202be72f48be81018bc5"
server_key_iv: "ddafe310c8bf16fb730f2081bd6e671d"
server_key_version: "encryptionkey@0"
I20260806 06:34:57.927150 3924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:34:57.942966 3924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:34:57.943212 3924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:34:57.943334 3924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:34:57.946827 3924 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.126:38355
I20260806 06:34:57.946862 4017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.126:38355 every 8 connection(s)
I20260806 06:34:57.947132 3924 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:34:57.948000 4019 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:34:57.949836 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 3924
I20260806 06:34:57.949939 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/master-0/wal/instance
I20260806 06:34:57.950079 4019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2eb8807d5b14ca2861ccdf953c733f1: Bootstrap starting.
I20260806 06:34:57.950110 7341 external_mini_cluster.cc:1428] Setting key 0ae184771bd30a01cd056294ab2ba1ef
I20260806 06:34:57.950736 4019 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e2eb8807d5b14ca2861ccdf953c733f1: Neither blocks nor log segments found. Creating new log.
I20260806 06:34:57.951040 4019 log.cc:824] T 00000000000000000000000000000000 P e2eb8807d5b14ca2861ccdf953c733f1: Log is configured to *not* fsync() on all Append() calls
I20260806 06:34:57.951737 4019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2eb8807d5b14ca2861ccdf953c733f1: No bootstrap required, opened a new log
I20260806 06:34:57.953840 4019 raft_consensus.cc:348] T 00000000000000000000000000000000 P e2eb8807d5b14ca2861ccdf953c733f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2eb8807d5b14ca2861ccdf953c733f1" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 38355 } }
I20260806 06:34:57.953991 4019 raft_consensus.cc:374] T 00000000000000000000000000000000 P e2eb8807d5b14ca2861ccdf953c733f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:34:57.954020 4019 raft_consensus.cc:729] T 00000000000000000000000000000000 P e2eb8807d5b14ca2861ccdf953c733f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2eb8807d5b14ca2861ccdf953c733f1, State: Initialized, Role: FOLLOWER
I20260806 06:34:57.954134 4019 consensus_queue.cc:260] T 00000000000000000000000000000000 P e2eb8807d5b14ca2861ccdf953c733f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2eb8807d5b14ca2861ccdf953c733f1" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 38355 } }
I20260806 06:34:57.954202 4019 raft_consensus.cc:388] T 00000000000000000000000000000000 P e2eb8807d5b14ca2861ccdf953c733f1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:34:57.954236 4019 raft_consensus.cc:482] T 00000000000000000000000000000000 P e2eb8807d5b14ca2861ccdf953c733f1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:34:57.954279 4019 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e2eb8807d5b14ca2861ccdf953c733f1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:34:57.954936 4019 raft_consensus.cc:504] T 00000000000000000000000000000000 P e2eb8807d5b14ca2861ccdf953c733f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2eb8807d5b14ca2861ccdf953c733f1" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 38355 } }
I20260806 06:34:57.955049 4019 leader_election.cc:302] T 00000000000000000000000000000000 P e2eb8807d5b14ca2861ccdf953c733f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e2eb8807d5b14ca2861ccdf953c733f1; no voters:
I20260806 06:34:57.955207 4019 leader_election.cc:288] T 00000000000000000000000000000000 P e2eb8807d5b14ca2861ccdf953c733f1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:34:57.955240 4026 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e2eb8807d5b14ca2861ccdf953c733f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:34:57.955394 4026 raft_consensus.cc:686] T 00000000000000000000000000000000 P e2eb8807d5b14ca2861ccdf953c733f1 [term 1 LEADER]: Becoming Leader. State: Replica: e2eb8807d5b14ca2861ccdf953c733f1, State: Running, Role: LEADER
I20260806 06:34:57.955458 4019 sys_catalog.cc:564] T 00000000000000000000000000000000 P e2eb8807d5b14ca2861ccdf953c733f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:34:57.955520 4026 consensus_queue.cc:237] T 00000000000000000000000000000000 P e2eb8807d5b14ca2861ccdf953c733f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2eb8807d5b14ca2861ccdf953c733f1" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 38355 } }
I20260806 06:34:57.955806 4022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:57.950559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33702 (local address 127.7.43.126:38355)
0806 06:34:57.950778 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:57.950786 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:57.950819 (+ 33us) server_negotiation.cc:407] Connection header received
0806 06:34:57.951333 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:57.951340 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:57.951395 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:57.951530 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:34:57.951725 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:57.953082 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:57.953807 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:57.954399 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:57.954510 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:57.955055 (+ 545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:34:57.955060 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:34:57.955060 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:34:57.955257 (+ 197us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:34:57.955298 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:34:57.955314 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:34:57.955381 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:57.955549 (+ 168us) server_negotiation.cc:299] Negotiation successful
0806 06:34:57.955624 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":82,"threads_started":1}
I20260806 06:34:57.955829 4026 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2eb8807d5b14ca2861ccdf953c733f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e2eb8807d5b14ca2861ccdf953c733f1. Latest consensus state: current_term: 1 leader_uuid: "e2eb8807d5b14ca2861ccdf953c733f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2eb8807d5b14ca2861ccdf953c733f1" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 38355 } } }
I20260806 06:34:57.955891 4026 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2eb8807d5b14ca2861ccdf953c733f1 [sys.catalog]: This master's current role is: LEADER
I20260806 06:34:57.956027 4027 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2eb8807d5b14ca2861ccdf953c733f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e2eb8807d5b14ca2861ccdf953c733f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2eb8807d5b14ca2861ccdf953c733f1" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 38355 } } }
I20260806 06:34:57.956106 4027 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2eb8807d5b14ca2861ccdf953c733f1 [sys.catalog]: This master's current role is: LEADER
I20260806 06:34:57.956715 4034 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:34:57.957131 4034 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:34:57.958356 4034 catalog_manager.cc:1347] Generated new cluster ID: 44a7a10c32f0422582cf05a51be9054f
I20260806 06:34:57.958406 4034 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:34:57.972384 4034 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:34:57.972842 4034 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:34:57.979512 4034 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e2eb8807d5b14ca2861ccdf953c733f1: Generated new TSK 0
I20260806 06:34:57.979651 4034 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:34:57.986519 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.65:0
--local_ip_for_outbound_sockets=127.7.43.65
--webserver_interface=127.7.43.65
--webserver_port=0
--tserver_master_addrs=127.7.43.126:38355
--builtin_ntp_servers=127.7.43.84:42539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 06:34:58.065330 4061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:34:58.065507 4061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:34:58.065527 4061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:34:58.066646 4061 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:34:58.066687 4061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:34:58.066711 4061 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.65
I20260806 06:34:58.067986 4061 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:42539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.43.65
--webserver_port=0
--tserver_master_addrs=127.7.43.126:38355
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4061
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.65
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:34:58.068183 4061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:34:58.068439 4061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:34:58.073802 4061 server_base.cc:1034] running on GCE node
W20260806 06:34:58.073803 4067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:34:58.073776 4072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:34:58.073788 4068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:34:58.074085 4061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:34:58.074290 4061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:34:58.075449 4061 hybrid_clock.cc:648] HybridClock initialized: now 1785998098075437 us; error 29 us; skew 500 ppm
I20260806 06:34:58.076500 4061 webserver.cc:460] Webserver started at http://127.7.43.65:43525/ using document root <none> and password file <none>
I20260806 06:34:58.076694 4061 fs_manager.cc:359] Metadata directory not provided
I20260806 06:34:58.076736 4061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:34:58.076866 4061 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:34:58.077816 4061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/ts-0/data/instance:
uuid: "de4d73cb2c5049519be190014ea404a7"
format_stamp: "Formatted at 2026-08-06 06:34:58 on dist-test-slave-6zbq"
server_key: "0eee778a9269cf6598c85d62205925e9"
server_key_iv: "3c533d48cb64901605076a959df910da"
server_key_version: "encryptionkey@0"
I20260806 06:34:58.078122 4061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/ts-0/wal/instance:
uuid: "de4d73cb2c5049519be190014ea404a7"
format_stamp: "Formatted at 2026-08-06 06:34:58 on dist-test-slave-6zbq"
server_key: "0eee778a9269cf6598c85d62205925e9"
server_key_iv: "3c533d48cb64901605076a959df910da"
server_key_version: "encryptionkey@0"
I20260806 06:34:58.079316 4061 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:34:58.079986 4078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:58.080127 4061 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:34:58.080209 4061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/ts-0/wal
uuid: "de4d73cb2c5049519be190014ea404a7"
format_stamp: "Formatted at 2026-08-06 06:34:58 on dist-test-slave-6zbq"
server_key: "0eee778a9269cf6598c85d62205925e9"
server_key_iv: "3c533d48cb64901605076a959df910da"
server_key_version: "encryptionkey@0"
I20260806 06:34:58.080278 4061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:34:58.094822 4061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:34:58.095052 4061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:34:58.095168 4061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:34:58.095358 4061 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:34:58.095635 4061 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:34:58.095667 4061 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:58.095692 4061 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:34:58.095717 4061 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:58.101032 4061 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.65:42521
I20260806 06:34:58.101104 4231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.65:42521 every 8 connection(s)
I20260806 06:34:58.101373 4061 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/ts-0/data/info.pb
I20260806 06:34:58.105976 4022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:58.102018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:54189 (local address 127.7.43.126:38355)
0806 06:34:58.102082 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:58.102084 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:58.102294 (+ 210us) server_negotiation.cc:407] Connection header received
0806 06:34:58.102792 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:58.102795 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:58.102808 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:58.102872 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:34:58.103546 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:58.104026 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:58.104905 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:58.105091 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:58.105171 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:58.105788 (+ 617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:34:58.105789 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:34:58.105789 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:34:58.105793 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:34:58.105805 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:34:58.105808 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:34:58.105845 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:58.105900 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 06:34:58.105920 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 06:34:58.106149 4235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:58.101979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38355 (local address 127.7.43.65:54189)
0806 06:34:58.102188 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:58.102219 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:58.102681 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:58.102996 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:58.102999 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:58.103007 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:34:58.103453 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:58.103457 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:58.104089 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:58.104090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:58.104828 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:58.104830 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:58.105173 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:58.105174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:58.105271 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:58.105671 (+ 400us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:34:58.105672 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:34:58.105696 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:34:58.105717 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:34:58.105847 (+ 130us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:34:58.105849 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:34:58.105851 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:34:58.105894 (+ 43us) client_negotiation.cc:240] Negotiation successful
0806 06:34:58.105965 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":78,"threads_started":1}
I20260806 06:34:58.106582 4233 heartbeater.cc:347] Connected to a master server at 127.7.43.126:38355
I20260806 06:34:58.106662 4233 heartbeater.cc:464] Registering TS with master...
I20260806 06:34:58.106823 4233 heartbeater.cc:511] Master 127.7.43.126:38355 requested a full tablet report, sending...
I20260806 06:34:58.107270 3972 ts_manager.cc:194] Registered new tserver with Master: de4d73cb2c5049519be190014ea404a7 (127.7.43.65:42521)
I20260806 06:34:58.107836 3972 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.43.65:54189
I20260806 06:34:58.111512 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 4061
I20260806 06:34:58.111593 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/ts-0/wal/instance
I20260806 06:34:58.111745 7341 external_mini_cluster.cc:1428] Setting key 24c45da0b843e54fb2e277480a730fc3
I20260806 06:34:58.112540 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.66:0
--local_ip_for_outbound_sockets=127.7.43.66
--webserver_interface=127.7.43.66
--webserver_port=0
--tserver_master_addrs=127.7.43.126:38355
--builtin_ntp_servers=127.7.43.84:42539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 06:34:58.191037 4239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:34:58.191284 4239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:34:58.191313 4239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:34:58.192562 4239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:34:58.192602 4239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:34:58.192625 4239 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.66
I20260806 06:34:58.193867 4239 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:42539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.43.66
--webserver_port=0
--tserver_master_addrs=127.7.43.126:38355
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4239
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.66
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:34:58.194089 4239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:34:58.194363 4239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:34:58.198871 4249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:34:58.198972 4239 server_base.cc:1034] running on GCE node
W20260806 06:34:58.198884 4246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:34:58.198884 4247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:34:58.199272 4239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:34:58.199502 4239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:34:58.200639 4239 hybrid_clock.cc:648] HybridClock initialized: now 1785998098200625 us; error 27 us; skew 500 ppm
I20260806 06:34:58.201761 4239 webserver.cc:460] Webserver started at http://127.7.43.66:45559/ using document root <none> and password file <none>
I20260806 06:34:58.201967 4239 fs_manager.cc:359] Metadata directory not provided
I20260806 06:34:58.202013 4239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:34:58.202132 4239 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:34:58.203235 4239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/ts-1/data/instance:
uuid: "0c95bf016a4644589a1a8f9c81aa397b"
format_stamp: "Formatted at 2026-08-06 06:34:58 on dist-test-slave-6zbq"
server_key: "3be61db4da73178c46826ddce4d0559f"
server_key_iv: "7215e7638e0a0daf92efb3bd4bafd381"
server_key_version: "encryptionkey@0"
I20260806 06:34:58.203557 4239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/ts-1/wal/instance:
uuid: "0c95bf016a4644589a1a8f9c81aa397b"
format_stamp: "Formatted at 2026-08-06 06:34:58 on dist-test-slave-6zbq"
server_key: "3be61db4da73178c46826ddce4d0559f"
server_key_iv: "7215e7638e0a0daf92efb3bd4bafd381"
server_key_version: "encryptionkey@0"
I20260806 06:34:58.204701 4239 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:34:58.205307 4255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:58.205454 4239 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:34:58.205534 4239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/ts-1/wal
uuid: "0c95bf016a4644589a1a8f9c81aa397b"
format_stamp: "Formatted at 2026-08-06 06:34:58 on dist-test-slave-6zbq"
server_key: "3be61db4da73178c46826ddce4d0559f"
server_key_iv: "7215e7638e0a0daf92efb3bd4bafd381"
server_key_version: "encryptionkey@0"
I20260806 06:34:58.205600 4239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:34:58.215996 4239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:34:58.216217 4239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:34:58.216325 4239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:34:58.216504 4239 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:34:58.216780 4239 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:34:58.216812 4239 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:58.216837 4239 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:34:58.216862 4239 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:58.221925 4239 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.66:39319
I20260806 06:34:58.221983 4415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.66:39319 every 8 connection(s)
I20260806 06:34:58.222236 4239 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/ts-1/data/info.pb
I20260806 06:34:58.226544 4022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:58.222873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:51323 (local address 127.7.43.126:38355)
0806 06:34:58.222933 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:58.222935 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:58.223121 (+ 186us) server_negotiation.cc:407] Connection header received
0806 06:34:58.223551 (+ 430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:58.223551 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:58.223562 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:58.223597 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:34:58.224170 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:58.224647 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:58.225512 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:58.225685 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:58.225726 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:58.226314 (+ 588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:34:58.226315 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:34:58.226316 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:34:58.226320 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:34:58.226333 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:34:58.226359 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:34:58.226413 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:58.226468 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 06:34:58.226488 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 06:34:58.226727 4420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:58.222850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38355 (local address 127.7.43.66:51323)
0806 06:34:58.223013 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:58.223049 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:58.223464 (+ 415us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:58.223659 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:58.223662 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:58.223669 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:34:58.224113 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:58.224118 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:58.224705 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:58.224705 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:58.225454 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:58.225456 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:58.225735 (+ 279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:58.225736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:58.225831 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:58.226206 (+ 375us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:34:58.226206 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:34:58.226230 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:34:58.226249 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:34:58.226415 (+ 166us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:34:58.226417 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:34:58.226419 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:34:58.226462 (+ 43us) client_negotiation.cc:240] Negotiation successful
0806 06:34:58.226532 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":74,"threads_started":1}
I20260806 06:34:58.227146 4416 heartbeater.cc:347] Connected to a master server at 127.7.43.126:38355
I20260806 06:34:58.227227 4416 heartbeater.cc:464] Registering TS with master...
I20260806 06:34:58.227293 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 4239
I20260806 06:34:58.227396 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/ts-1/wal/instance
I20260806 06:34:58.227403 4416 heartbeater.cc:511] Master 127.7.43.126:38355 requested a full tablet report, sending...
I20260806 06:34:58.227591 7341 external_mini_cluster.cc:1428] Setting key 11cc379ef0593da66ca847f6cefa7fb5
I20260806 06:34:58.227769 3972 ts_manager.cc:194] Registered new tserver with Master: 0c95bf016a4644589a1a8f9c81aa397b (127.7.43.66:39319)
I20260806 06:34:58.228226 3972 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.43.66:51323
I20260806 06:34:58.228602 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.43.67:0
--local_ip_for_outbound_sockets=127.7.43.67
--webserver_interface=127.7.43.67
--webserver_port=0
--tserver_master_addrs=127.7.43.126:38355
--builtin_ntp_servers=127.7.43.84:42539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 06:34:58.307893 4421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:34:58.308070 4421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:34:58.308090 4421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:34:58.309207 4421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:34:58.309244 4421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:34:58.309267 4421 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.43.67
I20260806 06:34:58.310609 4421 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:42539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.43.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.43.67
--webserver_port=0
--tserver_master_addrs=127.7.43.126:38355
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4421
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.43.67
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:34:58.310806 4421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:34:58.311048 4421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:34:58.315587 4434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:34:58.315640 4428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:34:58.315621 4431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:34:58.315739 4421 server_base.cc:1034] running on GCE node
I20260806 06:34:58.315968 4421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:34:58.316172 4421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:34:58.317318 4421 hybrid_clock.cc:648] HybridClock initialized: now 1785998098317298 us; error 31 us; skew 500 ppm
I20260806 06:34:58.318423 4421 webserver.cc:460] Webserver started at http://127.7.43.67:37771/ using document root <none> and password file <none>
I20260806 06:34:58.318619 4421 fs_manager.cc:359] Metadata directory not provided
I20260806 06:34:58.318665 4421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:34:58.318781 4421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:34:58.319737 4421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/ts-2/data/instance:
uuid: "c96152fcc2c04079a273caa13e9aa3e6"
format_stamp: "Formatted at 2026-08-06 06:34:58 on dist-test-slave-6zbq"
server_key: "a2f841c75fbbdc01a35cbaeb8a8aba79"
server_key_iv: "dd0ad4371b7f0abab1e7984168e6bbe3"
server_key_version: "encryptionkey@0"
I20260806 06:34:58.320042 4421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/ts-2/wal/instance:
uuid: "c96152fcc2c04079a273caa13e9aa3e6"
format_stamp: "Formatted at 2026-08-06 06:34:58 on dist-test-slave-6zbq"
server_key: "a2f841c75fbbdc01a35cbaeb8a8aba79"
server_key_iv: "dd0ad4371b7f0abab1e7984168e6bbe3"
server_key_version: "encryptionkey@0"
I20260806 06:34:58.321187 4421 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:34:58.321769 4440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:58.321919 4421 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:34:58.322000 4421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/ts-2/wal
uuid: "c96152fcc2c04079a273caa13e9aa3e6"
format_stamp: "Formatted at 2026-08-06 06:34:58 on dist-test-slave-6zbq"
server_key: "a2f841c75fbbdc01a35cbaeb8a8aba79"
server_key_iv: "dd0ad4371b7f0abab1e7984168e6bbe3"
server_key_version: "encryptionkey@0"
I20260806 06:34:58.322058 4421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:34:58.335374 4421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:34:58.335592 4421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:34:58.335701 4421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:34:58.335886 4421 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:34:58.336169 4421 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:34:58.336200 4421 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:58.336226 4421 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:34:58.336243 4421 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:58.341547 4421 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.67:37727
I20260806 06:34:58.341581 4606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.67:37727 every 8 connection(s)
I20260806 06:34:58.341836 4421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/ts-2/data/info.pb
I20260806 06:34:58.343485 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 4421
I20260806 06:34:58.343580 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/ts-2/wal/instance
I20260806 06:34:58.343758 7341 external_mini_cluster.cc:1428] Setting key 88d26bed7591f62b897690c1a0a09053
I20260806 06:34:58.346593 4022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:58.342500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.67:58601 (local address 127.7.43.126:38355)
0806 06:34:58.342557 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:58.342559 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:58.342766 (+ 207us) server_negotiation.cc:407] Connection header received
0806 06:34:58.343208 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:58.343209 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:58.343222 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:58.343272 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:34:58.343928 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:58.344521 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:58.345427 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:58.345633 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:58.345686 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:58.346298 (+ 612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:34:58.346300 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:34:58.346300 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:34:58.346304 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:34:58.346319 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:34:58.346323 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:34:58.346388 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:58.346492 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 06:34:58.346517 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 06:34:58.346766 4610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:58.342495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:38355 (local address 127.7.43.67:58601)
0806 06:34:58.342665 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:58.342693 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:58.343106 (+ 413us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:58.343332 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:58.343336 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:58.343345 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:34:58.343842 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:58.343848 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:58.344608 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:58.344609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:58.345344 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:58.345346 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:58.345698 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:58.345699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:58.345827 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:58.346177 (+ 350us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:34:58.346177 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:34:58.346200 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:34:58.346220 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:34:58.346420 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:34:58.346425 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:34:58.346428 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:34:58.346486 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 06:34:58.346585 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":77,"threads_started":1}
I20260806 06:34:58.347144 4607 heartbeater.cc:347] Connected to a master server at 127.7.43.126:38355
I20260806 06:34:58.347239 4607 heartbeater.cc:464] Registering TS with master...
I20260806 06:34:58.347404 4607 heartbeater.cc:511] Master 127.7.43.126:38355 requested a full tablet report, sending...
I20260806 06:34:58.347708 3972 ts_manager.cc:194] Registered new tserver with Master: c96152fcc2c04079a273caa13e9aa3e6 (127.7.43.67:37727)
I20260806 06:34:58.348057 3972 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.43.67:58601
I20260806 06:34:58.355207 7341 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:34:58.359040 4022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:58.356432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33714 (local address 127.7.43.126:38355)
0806 06:34:58.356477 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:58.356479 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:58.356613 (+ 134us) server_negotiation.cc:407] Connection header received
0806 06:34:58.356672 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:58.356673 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:58.356682 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:58.356727 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:34:58.356865 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:58.357306 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:58.358011 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:58.358207 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:58.358258 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:58.358842 (+ 584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:34:58.358843 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:34:58.358844 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:34:58.358848 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:34:58.358860 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:34:58.358864 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:34:58.358905 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:58.358952 (+ 47us) server_negotiation.cc:299] Negotiation successful
0806 06:34:58.358972 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 06:34:58.360297 3972 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33714:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:34:58.361693 3972 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:34:58.367230 4625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:58.363725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.66:39319 (local address 127.0.0.1:51534)
0806 06:34:58.363977 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:58.363985 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:58.364030 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:58.364275 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:58.364276 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:58.364278 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:34:58.364311 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:58.364312 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:58.365510 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:58.365511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:58.366265 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:58.366267 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:58.366962 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:58.366962 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:58.367037 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:58.367051 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 06:34:58.367117 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 06:34:58.367162 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":59,"threads_started":1}
I20260806 06:34:58.367259 4628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:58.363823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51534 (local address 127.7.43.66:39319)
0806 06:34:58.363990 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:58.363996 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:58.364022 (+ 26us) server_negotiation.cc:407] Connection header received
0806 06:34:58.364118 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:58.364120 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:58.364137 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:58.364205 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:34:58.364384 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:58.365444 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:58.366368 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:58.366882 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:58.366947 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:58.367029 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:58.367134 (+ 105us) server_negotiation.cc:299] Negotiation successful
0806 06:34:58.367168 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":87,"threads_started":1}
I20260806 06:34:58.367489 4619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:58.363537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:42521 (local address 127.0.0.1:48142)
0806 06:34:58.363785 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:58.363824 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:58.363882 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:58.364170 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:58.364171 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:58.364173 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:34:58.364217 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:58.364218 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:58.365163 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:58.365164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:58.366395 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:58.366398 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:58.367284 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:58.367285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:58.367348 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:58.367351 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:34:58.367380 (+ 29us) client_negotiation.cc:240] Negotiation successful
0806 06:34:58.367418 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"mutex_wait_us":3,"thread_start_us":51,"threads_started":1}
I20260806 06:34:58.367642 4623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:58.363614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53684 (local address 127.7.43.67:37727)
0806 06:34:58.363859 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:58.363863 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:58.363873 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:34:58.363986 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:58.363990 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:58.364004 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:58.364075 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:34:58.364232 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:58.365300 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:58.366517 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:58.367185 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:58.367252 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:58.367329 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:58.367526 (+ 197us) server_negotiation.cc:299] Negotiation successful
0806 06:34:58.367561 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":71,"threads_started":1}
I20260806 06:34:58.367588 4622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:58.363558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.67:37727 (local address 127.0.0.1:53684)
0806 06:34:58.363820 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:58.363830 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:58.363883 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:58.364092 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:58.364094 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:58.364096 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:34:58.364153 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:58.364156 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:58.365386 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:58.365387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:58.366442 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:58.366444 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:58.367358 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:58.367359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:58.367409 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:58.367411 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:34:58.367453 (+ 42us) client_negotiation.cc:240] Negotiation successful
0806 06:34:58.367484 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":110,"threads_started":1}
I20260806 06:34:58.368136 4624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:58.363646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48142 (local address 127.7.43.65:42521)
0806 06:34:58.364038 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:58.364042 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:58.364052 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:34:58.364086 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:58.364088 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:58.364101 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:58.364170 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:34:58.364294 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:58.365087 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:58.366491 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:58.367191 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:58.367888 (+ 697us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:58.367982 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:58.368006 (+ 24us) server_negotiation.cc:299] Negotiation successful
0806 06:34:58.368044 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":80,"threads_started":1}
I20260806 06:34:58.368127 4319 tablet_service.cc:1467] Processing CreateTablet for tablet a15043ef2bc546dcbb1622d7013e5d09 (DEFAULT_TABLE table=test-table [id=e5d38091b861416e96e6e7c8745c64e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:34:58.368484 4319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a15043ef2bc546dcbb1622d7013e5d09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:34:58.368593 4506 tablet_service.cc:1467] Processing CreateTablet for tablet a15043ef2bc546dcbb1622d7013e5d09 (DEFAULT_TABLE table=test-table [id=e5d38091b861416e96e6e7c8745c64e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:34:58.368721 4140 tablet_service.cc:1467] Processing CreateTablet for tablet a15043ef2bc546dcbb1622d7013e5d09 (DEFAULT_TABLE table=test-table [id=e5d38091b861416e96e6e7c8745c64e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:34:58.368881 4506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a15043ef2bc546dcbb1622d7013e5d09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:34:58.369007 4140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a15043ef2bc546dcbb1622d7013e5d09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:34:58.371531 4629 tablet_bootstrap.cc:492] T a15043ef2bc546dcbb1622d7013e5d09 P de4d73cb2c5049519be190014ea404a7: Bootstrap starting.
I20260806 06:34:58.371923 4631 tablet_bootstrap.cc:492] T a15043ef2bc546dcbb1622d7013e5d09 P 0c95bf016a4644589a1a8f9c81aa397b: Bootstrap starting.
I20260806 06:34:58.372118 4629 tablet_bootstrap.cc:654] T a15043ef2bc546dcbb1622d7013e5d09 P de4d73cb2c5049519be190014ea404a7: Neither blocks nor log segments found. Creating new log.
I20260806 06:34:58.372373 4629 log.cc:824] T a15043ef2bc546dcbb1622d7013e5d09 P de4d73cb2c5049519be190014ea404a7: Log is configured to *not* fsync() on all Append() calls
I20260806 06:34:58.372449 4631 tablet_bootstrap.cc:654] T a15043ef2bc546dcbb1622d7013e5d09 P 0c95bf016a4644589a1a8f9c81aa397b: Neither blocks nor log segments found. Creating new log.
I20260806 06:34:58.372748 4631 log.cc:824] T a15043ef2bc546dcbb1622d7013e5d09 P 0c95bf016a4644589a1a8f9c81aa397b: Log is configured to *not* fsync() on all Append() calls
I20260806 06:34:58.373039 4630 tablet_bootstrap.cc:492] T a15043ef2bc546dcbb1622d7013e5d09 P c96152fcc2c04079a273caa13e9aa3e6: Bootstrap starting.
I20260806 06:34:58.373149 4629 tablet_bootstrap.cc:492] T a15043ef2bc546dcbb1622d7013e5d09 P de4d73cb2c5049519be190014ea404a7: No bootstrap required, opened a new log
I20260806 06:34:58.373222 4629 ts_tablet_manager.cc:1397] T a15043ef2bc546dcbb1622d7013e5d09 P de4d73cb2c5049519be190014ea404a7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:34:58.373467 4631 tablet_bootstrap.cc:492] T a15043ef2bc546dcbb1622d7013e5d09 P 0c95bf016a4644589a1a8f9c81aa397b: No bootstrap required, opened a new log
I20260806 06:34:58.373538 4631 ts_tablet_manager.cc:1397] T a15043ef2bc546dcbb1622d7013e5d09 P 0c95bf016a4644589a1a8f9c81aa397b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:34:58.373611 4630 tablet_bootstrap.cc:654] T a15043ef2bc546dcbb1622d7013e5d09 P c96152fcc2c04079a273caa13e9aa3e6: Neither blocks nor log segments found. Creating new log.
I20260806 06:34:58.373905 4630 log.cc:824] T a15043ef2bc546dcbb1622d7013e5d09 P c96152fcc2c04079a273caa13e9aa3e6: Log is configured to *not* fsync() on all Append() calls
I20260806 06:34:58.374653 4630 tablet_bootstrap.cc:492] T a15043ef2bc546dcbb1622d7013e5d09 P c96152fcc2c04079a273caa13e9aa3e6: No bootstrap required, opened a new log
I20260806 06:34:58.374720 4630 ts_tablet_manager.cc:1397] T a15043ef2bc546dcbb1622d7013e5d09 P c96152fcc2c04079a273caa13e9aa3e6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:34:58.375281 4629 raft_consensus.cc:348] T a15043ef2bc546dcbb1622d7013e5d09 P de4d73cb2c5049519be190014ea404a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4d73cb2c5049519be190014ea404a7" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 42521 } } peers { permanent_uuid: "0c95bf016a4644589a1a8f9c81aa397b" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 39319 } } peers { permanent_uuid: "c96152fcc2c04079a273caa13e9aa3e6" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 37727 } }
I20260806 06:34:58.375403 4629 raft_consensus.cc:374] T a15043ef2bc546dcbb1622d7013e5d09 P de4d73cb2c5049519be190014ea404a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:34:58.375434 4629 raft_consensus.cc:729] T a15043ef2bc546dcbb1622d7013e5d09 P de4d73cb2c5049519be190014ea404a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4d73cb2c5049519be190014ea404a7, State: Initialized, Role: FOLLOWER
I20260806 06:34:58.375561 4629 consensus_queue.cc:260] T a15043ef2bc546dcbb1622d7013e5d09 P de4d73cb2c5049519be190014ea404a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4d73cb2c5049519be190014ea404a7" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 42521 } } peers { permanent_uuid: "0c95bf016a4644589a1a8f9c81aa397b" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 39319 } } peers { permanent_uuid: "c96152fcc2c04079a273caa13e9aa3e6" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 37727 } }
I20260806 06:34:58.375797 4629 ts_tablet_manager.cc:1428] T a15043ef2bc546dcbb1622d7013e5d09 P de4d73cb2c5049519be190014ea404a7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:34:58.375917 4233 heartbeater.cc:503] Master 127.7.43.126:38355 was elected leader, sending a full tablet report...
I20260806 06:34:58.376013 4631 raft_consensus.cc:348] T a15043ef2bc546dcbb1622d7013e5d09 P 0c95bf016a4644589a1a8f9c81aa397b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4d73cb2c5049519be190014ea404a7" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 42521 } } peers { permanent_uuid: "0c95bf016a4644589a1a8f9c81aa397b" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 39319 } } peers { permanent_uuid: "c96152fcc2c04079a273caa13e9aa3e6" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 37727 } }
I20260806 06:34:58.376156 4631 raft_consensus.cc:374] T a15043ef2bc546dcbb1622d7013e5d09 P 0c95bf016a4644589a1a8f9c81aa397b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:34:58.376236 4631 raft_consensus.cc:729] T a15043ef2bc546dcbb1622d7013e5d09 P 0c95bf016a4644589a1a8f9c81aa397b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c95bf016a4644589a1a8f9c81aa397b, State: Initialized, Role: FOLLOWER
I20260806 06:34:58.376382 4631 consensus_queue.cc:260] T a15043ef2bc546dcbb1622d7013e5d09 P 0c95bf016a4644589a1a8f9c81aa397b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4d73cb2c5049519be190014ea404a7" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 42521 } } peers { permanent_uuid: "0c95bf016a4644589a1a8f9c81aa397b" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 39319 } } peers { permanent_uuid: "c96152fcc2c04079a273caa13e9aa3e6" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 37727 } }
I20260806 06:34:58.376636 4631 ts_tablet_manager.cc:1428] T a15043ef2bc546dcbb1622d7013e5d09 P 0c95bf016a4644589a1a8f9c81aa397b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:34:58.376688 4416 heartbeater.cc:503] Master 127.7.43.126:38355 was elected leader, sending a full tablet report...
I20260806 06:34:58.377393 4630 raft_consensus.cc:348] T a15043ef2bc546dcbb1622d7013e5d09 P c96152fcc2c04079a273caa13e9aa3e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4d73cb2c5049519be190014ea404a7" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 42521 } } peers { permanent_uuid: "0c95bf016a4644589a1a8f9c81aa397b" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 39319 } } peers { permanent_uuid: "c96152fcc2c04079a273caa13e9aa3e6" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 37727 } }
I20260806 06:34:58.377537 4630 raft_consensus.cc:374] T a15043ef2bc546dcbb1622d7013e5d09 P c96152fcc2c04079a273caa13e9aa3e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:34:58.377578 4630 raft_consensus.cc:729] T a15043ef2bc546dcbb1622d7013e5d09 P c96152fcc2c04079a273caa13e9aa3e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c96152fcc2c04079a273caa13e9aa3e6, State: Initialized, Role: FOLLOWER
I20260806 06:34:58.377674 4630 consensus_queue.cc:260] T a15043ef2bc546dcbb1622d7013e5d09 P c96152fcc2c04079a273caa13e9aa3e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4d73cb2c5049519be190014ea404a7" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 42521 } } peers { permanent_uuid: "0c95bf016a4644589a1a8f9c81aa397b" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 39319 } } peers { permanent_uuid: "c96152fcc2c04079a273caa13e9aa3e6" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 37727 } }
I20260806 06:34:58.377872 4630 ts_tablet_manager.cc:1428] T a15043ef2bc546dcbb1622d7013e5d09 P c96152fcc2c04079a273caa13e9aa3e6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:34:58.377914 4607 heartbeater.cc:503] Master 127.7.43.126:38355 was elected leader, sending a full tablet report...
W20260806 06:34:58.472721 4419 tablet.cc:2369] T a15043ef2bc546dcbb1622d7013e5d09 P 0c95bf016a4644589a1a8f9c81aa397b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:34:58.473009 4636 raft_consensus.cc:482] T a15043ef2bc546dcbb1622d7013e5d09 P de4d73cb2c5049519be190014ea404a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:34:58.473109 4636 raft_consensus.cc:504] T a15043ef2bc546dcbb1622d7013e5d09 P de4d73cb2c5049519be190014ea404a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4d73cb2c5049519be190014ea404a7" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 42521 } } peers { permanent_uuid: "0c95bf016a4644589a1a8f9c81aa397b" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 39319 } } peers { permanent_uuid: "c96152fcc2c04079a273caa13e9aa3e6" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 37727 } }
I20260806 06:34:58.473440 4636 leader_election.cc:288] T a15043ef2bc546dcbb1622d7013e5d09 P de4d73cb2c5049519be190014ea404a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0c95bf016a4644589a1a8f9c81aa397b (127.7.43.66:39319), c96152fcc2c04079a273caa13e9aa3e6 (127.7.43.67:37727)
I20260806 06:34:58.476824 4638 raft_consensus.cc:482] T a15043ef2bc546dcbb1622d7013e5d09 P 0c95bf016a4644589a1a8f9c81aa397b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:34:58.476917 4638 raft_consensus.cc:504] T a15043ef2bc546dcbb1622d7013e5d09 P 0c95bf016a4644589a1a8f9c81aa397b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4d73cb2c5049519be190014ea404a7" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 42521 } } peers { permanent_uuid: "0c95bf016a4644589a1a8f9c81aa397b" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 39319 } } peers { permanent_uuid: "c96152fcc2c04079a273caa13e9aa3e6" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 37727 } }
I20260806 06:34:58.477012 4235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:58.473573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.66:39319 (local address 127.7.43.65:37607)
0806 06:34:58.473621 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:58.473632 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:58.473673 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:58.473813 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:58.473814 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:58.473816 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:34:58.473888 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:58.473890 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:58.474781 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:58.474782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:58.476037 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:58.476040 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:58.476769 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:58.476770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:58.476848 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:58.476854 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 06:34:58.476899 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 06:34:58.476943 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26,"mutex_wait_us":31}
I20260806 06:34:58.477041 4623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:58.473736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:50155 (local address 127.7.43.67:37727)
0806 06:34:58.473795 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:58.473797 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:58.473806 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:34:58.473872 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:58.473872 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:58.473879 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:58.473919 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:34:58.474038 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:58.474733 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:58.476113 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:58.476667 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:58.476735 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:58.476751 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:58.476930 (+ 179us) server_negotiation.cc:299] Negotiation successful
0806 06:34:58.476988 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 06:34:58.477202 4628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:58.473645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.65:37607 (local address 127.7.43.66:39319)
0806 06:34:58.473699 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:58.473702 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:58.473710 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:34:58.473738 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:58.473738 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:58.473747 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:58.473809 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:34:58.473948 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:58.474710 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:58.476113 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:58.476653 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:58.476721 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:58.476731 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:58.477104 (+ 373us) server_negotiation.cc:299] Negotiation successful
0806 06:34:58.477145 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 06:34:58.477210 4644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:58.473614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.67:37727 (local address 127.7.43.65:50155)
0806 06:34:58.473768 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:58.473775 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:58.473812 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:58.473931 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:58.473932 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:58.473934 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:34:58.473986 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:58.473987 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:58.474796 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:58.474797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:58.476037 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:58.476040 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:58.476792 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:58.476792 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:58.476849 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:58.476855 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 06:34:58.477103 (+ 248us) client_negotiation.cc:240] Negotiation successful
0806 06:34:58.477139 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":9,"thread_start_us":84,"threads_started":1}
I20260806 06:34:58.477219 4638 leader_election.cc:288] T a15043ef2bc546dcbb1622d7013e5d09 P 0c95bf016a4644589a1a8f9c81aa397b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de4d73cb2c5049519be190014ea404a7 (127.7.43.65:42521), c96152fcc2c04079a273caa13e9aa3e6 (127.7.43.67:37727)
I20260806 06:34:58.477497 4343 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a15043ef2bc546dcbb1622d7013e5d09" candidate_uuid: "de4d73cb2c5049519be190014ea404a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c95bf016a4644589a1a8f9c81aa397b" is_pre_election: true
I20260806 06:34:58.477572 4343 raft_consensus.cc:2454] T a15043ef2bc546dcbb1622d7013e5d09 P 0c95bf016a4644589a1a8f9c81aa397b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4d73cb2c5049519be190014ea404a7 in term 0.
I20260806 06:34:58.477682 4534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a15043ef2bc546dcbb1622d7013e5d09" candidate_uuid: "de4d73cb2c5049519be190014ea404a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c96152fcc2c04079a273caa13e9aa3e6" is_pre_election: true
I20260806 06:34:58.477775 4081 leader_election.cc:302] T a15043ef2bc546dcbb1622d7013e5d09 P de4d73cb2c5049519be190014ea404a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c95bf016a4644589a1a8f9c81aa397b, de4d73cb2c5049519be190014ea404a7; no voters:
I20260806 06:34:58.477824 4534 raft_consensus.cc:2454] T a15043ef2bc546dcbb1622d7013e5d09 P c96152fcc2c04079a273caa13e9aa3e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4d73cb2c5049519be190014ea404a7 in term 0.
I20260806 06:34:58.477901 4636 raft_consensus.cc:2781] T a15043ef2bc546dcbb1622d7013e5d09 P de4d73cb2c5049519be190014ea404a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:34:58.477957 4636 raft_consensus.cc:482] T a15043ef2bc546dcbb1622d7013e5d09 P de4d73cb2c5049519be190014ea404a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:34:58.477980 4636 raft_consensus.cc:3037] T a15043ef2bc546dcbb1622d7013e5d09 P de4d73cb2c5049519be190014ea404a7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:34:58.478861 4636 raft_consensus.cc:504] T a15043ef2bc546dcbb1622d7013e5d09 P de4d73cb2c5049519be190014ea404a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4d73cb2c5049519be190014ea404a7" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 42521 } } peers { permanent_uuid: "0c95bf016a4644589a1a8f9c81aa397b" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 39319 } } peers { permanent_uuid: "c96152fcc2c04079a273caa13e9aa3e6" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 37727 } }
I20260806 06:34:58.478983 4636 leader_election.cc:288] T a15043ef2bc546dcbb1622d7013e5d09 P de4d73cb2c5049519be190014ea404a7 [CANDIDATE]: Term 1 election: Requested vote from peers 0c95bf016a4644589a1a8f9c81aa397b (127.7.43.66:39319), c96152fcc2c04079a273caa13e9aa3e6 (127.7.43.67:37727)
I20260806 06:34:58.479137 4534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a15043ef2bc546dcbb1622d7013e5d09" candidate_uuid: "de4d73cb2c5049519be190014ea404a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c96152fcc2c04079a273caa13e9aa3e6"
I20260806 06:34:58.479182 4343 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a15043ef2bc546dcbb1622d7013e5d09" candidate_uuid: "de4d73cb2c5049519be190014ea404a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c95bf016a4644589a1a8f9c81aa397b"
I20260806 06:34:58.479213 4534 raft_consensus.cc:3037] T a15043ef2bc546dcbb1622d7013e5d09 P c96152fcc2c04079a273caa13e9aa3e6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:34:58.479238 4343 raft_consensus.cc:3037] T a15043ef2bc546dcbb1622d7013e5d09 P 0c95bf016a4644589a1a8f9c81aa397b [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:34:58.480033 4343 raft_consensus.cc:2454] T a15043ef2bc546dcbb1622d7013e5d09 P 0c95bf016a4644589a1a8f9c81aa397b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4d73cb2c5049519be190014ea404a7 in term 1.
I20260806 06:34:58.480033 4534 raft_consensus.cc:2454] T a15043ef2bc546dcbb1622d7013e5d09 P c96152fcc2c04079a273caa13e9aa3e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4d73cb2c5049519be190014ea404a7 in term 1.
I20260806 06:34:58.480227 4081 leader_election.cc:302] T a15043ef2bc546dcbb1622d7013e5d09 P de4d73cb2c5049519be190014ea404a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c95bf016a4644589a1a8f9c81aa397b, de4d73cb2c5049519be190014ea404a7; no voters:
I20260806 06:34:58.480345 4636 raft_consensus.cc:2781] T a15043ef2bc546dcbb1622d7013e5d09 P de4d73cb2c5049519be190014ea404a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:34:58.480468 4636 raft_consensus.cc:686] T a15043ef2bc546dcbb1622d7013e5d09 P de4d73cb2c5049519be190014ea404a7 [term 1 LEADER]: Becoming Leader. State: Replica: de4d73cb2c5049519be190014ea404a7, State: Running, Role: LEADER
I20260806 06:34:58.480572 4636 consensus_queue.cc:237] T a15043ef2bc546dcbb1622d7013e5d09 P de4d73cb2c5049519be190014ea404a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4d73cb2c5049519be190014ea404a7" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 42521 } } peers { permanent_uuid: "0c95bf016a4644589a1a8f9c81aa397b" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 39319 } } peers { permanent_uuid: "c96152fcc2c04079a273caa13e9aa3e6" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 37727 } }
I20260806 06:34:58.480800 4623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:58.477427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:58071 (local address 127.7.43.67:37727)
0806 06:34:58.477459 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:58.477460 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:58.477868 (+ 408us) server_negotiation.cc:407] Connection header received
0806 06:34:58.477947 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:58.477948 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:58.477957 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:58.477984 (+ 27us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:34:58.478184 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:58.478654 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:58.479961 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:58.480462 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:58.480503 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:58.480510 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:58.480712 (+ 202us) server_negotiation.cc:299] Negotiation successful
0806 06:34:58.480741 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 06:34:58.480793 4420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:58.477303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:42521 (local address 127.7.43.66:38793)
0806 06:34:58.477356 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:58.477367 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:58.477403 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:58.478025 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:58.478026 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:58.478028 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:34:58.478082 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:58.478083 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:58.478860 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:58.478860 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:58.479901 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:58.479903 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:58.480583 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:58.480584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:58.480642 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:58.480645 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:34:58.480693 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 06:34:58.480730 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":11}
I20260806 06:34:58.480805 4624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:58.477695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.43.66:38793 (local address 127.7.43.65:42521)
0806 06:34:58.477899 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:58.477901 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:58.477907 (+ 6us) server_negotiation.cc:407] Connection header received
0806 06:34:58.477925 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:58.477926 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:58.477935 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:58.477983 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:34:58.478152 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:58.478782 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:58.479982 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:58.480495 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:58.480550 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:58.480558 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:58.480701 (+ 143us) server_negotiation.cc:299] Negotiation successful
0806 06:34:58.480755 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188}
I20260806 06:34:58.480983 4645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:58.477371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.67:37727 (local address 127.7.43.66:58071)
0806 06:34:58.477807 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:58.477817 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:34:58.477874 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:34:58.478041 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:34:58.478041 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:34:58.478043 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:34:58.478107 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:58.478108 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:58.478729 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:58.478730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:58.479859 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:34:58.479862 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:34:58.480567 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:34:58.480567 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:34:58.480634 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:58.480637 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:34:58.480887 (+ 250us) client_negotiation.cc:240] Negotiation successful
0806 06:34:58.480922 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":91,"threads_started":1}
I20260806 06:34:58.481192 4534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a15043ef2bc546dcbb1622d7013e5d09" candidate_uuid: "0c95bf016a4644589a1a8f9c81aa397b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c96152fcc2c04079a273caa13e9aa3e6" is_pre_election: true
I20260806 06:34:58.481290 4534 raft_consensus.cc:2379] T a15043ef2bc546dcbb1622d7013e5d09 P c96152fcc2c04079a273caa13e9aa3e6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0c95bf016a4644589a1a8f9c81aa397b in current term 1: Already voted for candidate de4d73cb2c5049519be190014ea404a7 in this term.
I20260806 06:34:58.481441 4161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a15043ef2bc546dcbb1622d7013e5d09" candidate_uuid: "0c95bf016a4644589a1a8f9c81aa397b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4d73cb2c5049519be190014ea404a7" is_pre_election: true
I20260806 06:34:58.481539 3972 catalog_manager.cc:5495] T a15043ef2bc546dcbb1622d7013e5d09 P de4d73cb2c5049519be190014ea404a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to de4d73cb2c5049519be190014ea404a7 (127.7.43.65). New cstate: current_term: 1 leader_uuid: "de4d73cb2c5049519be190014ea404a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4d73cb2c5049519be190014ea404a7" member_type: VOTER last_known_addr { host: "127.7.43.65" port: 42521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0c95bf016a4644589a1a8f9c81aa397b" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 39319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c96152fcc2c04079a273caa13e9aa3e6" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 37727 } health_report { overall_health: UNKNOWN } } }
I20260806 06:34:58.481693 4258 leader_election.cc:302] T a15043ef2bc546dcbb1622d7013e5d09 P 0c95bf016a4644589a1a8f9c81aa397b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0c95bf016a4644589a1a8f9c81aa397b; no voters: c96152fcc2c04079a273caa13e9aa3e6, de4d73cb2c5049519be190014ea404a7
I20260806 06:34:58.481851 4638 raft_consensus.cc:2726] T a15043ef2bc546dcbb1622d7013e5d09 P 0c95bf016a4644589a1a8f9c81aa397b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 06:34:58.508775 4624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:58.506846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48158 (local address 127.7.43.65:42521)
0806 06:34:58.506883 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:58.506885 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:58.506893 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:34:58.506942 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:58.506942 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:58.506955 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:58.507001 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:34:58.507090 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:58.507492 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:58.508225 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:58.508396 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:58.508443 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:58.508535 (+ 92us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:34:58.508617 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:34:58.508685 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:58.508701 (+ 16us) server_negotiation.cc:299] Negotiation successful
0806 06:34:58.508718 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 06:34:58.510249 4534 raft_consensus.cc:1261] T a15043ef2bc546dcbb1622d7013e5d09 P c96152fcc2c04079a273caa13e9aa3e6 [term 1 FOLLOWER]: Refusing update from remote peer de4d73cb2c5049519be190014ea404a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:34:58.510265 4343 raft_consensus.cc:1261] T a15043ef2bc546dcbb1622d7013e5d09 P 0c95bf016a4644589a1a8f9c81aa397b [term 1 FOLLOWER]: Refusing update from remote peer de4d73cb2c5049519be190014ea404a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:34:58.510524 4636 consensus_queue.cc:1035] T a15043ef2bc546dcbb1622d7013e5d09 P de4d73cb2c5049519be190014ea404a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c95bf016a4644589a1a8f9c81aa397b" member_type: VOTER last_known_addr { host: "127.7.43.66" port: 39319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:34:58.510661 4646 consensus_queue.cc:1035] T a15043ef2bc546dcbb1622d7013e5d09 P de4d73cb2c5049519be190014ea404a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c96152fcc2c04079a273caa13e9aa3e6" member_type: VOTER last_known_addr { host: "127.7.43.67" port: 37727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:34:58.512876 4651 mvcc.cc:205] Tried to move back new op lower bound from 7315448211495866368 to 7315448211376873472. Current Snapshot: MvccSnapshot[applied={T|T < 7315448211495866368}]
I20260806 06:34:58.513059 4652 mvcc.cc:205] Tried to move back new op lower bound from 7315448211495866368 to 7315448211376873472. Current Snapshot: MvccSnapshot[applied={T|T < 7315448211495866368}]
I20260806 06:34:58.513741 4653 mvcc.cc:205] Tried to move back new op lower bound from 7315448211495866368 to 7315448211376873472. Current Snapshot: MvccSnapshot[applied={T|T < 7315448211495866368 or (T in {7315448211495866368})}]
I20260806 06:34:58.515196 3972 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33714:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:34:58.515278 3972 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33714:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:34:58.515895 3972 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e2eb8807d5b14ca2861ccdf953c733f1: Sending DeleteTablet for 3 replicas of tablet a15043ef2bc546dcbb1622d7013e5d09
I20260806 06:34:58.516182 4140 tablet_service.cc:1514] Processing DeleteTablet for tablet a15043ef2bc546dcbb1622d7013e5d09 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:34:58 UTC) from {username='slave'} at 127.0.0.1:48142
I20260806 06:34:58.516217 4506 tablet_service.cc:1514] Processing DeleteTablet for tablet a15043ef2bc546dcbb1622d7013e5d09 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:34:58 UTC) from {username='slave'} at 127.0.0.1:53684
I20260806 06:34:58.516409 4319 tablet_service.cc:1514] Processing DeleteTablet for tablet a15043ef2bc546dcbb1622d7013e5d09 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:34:58 UTC) from {username='slave'} at 127.0.0.1:51534
I20260806 06:34:58.516441 4666 tablet_replica.cc:330] T a15043ef2bc546dcbb1622d7013e5d09 P c96152fcc2c04079a273caa13e9aa3e6: stopping tablet replica
I20260806 06:34:58.516525 4666 raft_consensus.cc:2229] T a15043ef2bc546dcbb1622d7013e5d09 P c96152fcc2c04079a273caa13e9aa3e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:34:58.516615 4666 raft_consensus.cc:2258] T a15043ef2bc546dcbb1622d7013e5d09 P c96152fcc2c04079a273caa13e9aa3e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:34:58.516683 4667 tablet_replica.cc:330] T a15043ef2bc546dcbb1622d7013e5d09 P 0c95bf016a4644589a1a8f9c81aa397b: stopping tablet replica
I20260806 06:34:58.516742 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 4061
I20260806 06:34:58.516884 4667 raft_consensus.cc:2229] T a15043ef2bc546dcbb1622d7013e5d09 P 0c95bf016a4644589a1a8f9c81aa397b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:34:58.517030 4667 raft_consensus.cc:2258] T a15043ef2bc546dcbb1622d7013e5d09 P 0c95bf016a4644589a1a8f9c81aa397b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:34:58.517059 4666 ts_tablet_manager.cc:1905] T a15043ef2bc546dcbb1622d7013e5d09 P c96152fcc2c04079a273caa13e9aa3e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:34:58.518424 4666 ts_tablet_manager.cc:1918] T a15043ef2bc546dcbb1622d7013e5d09 P c96152fcc2c04079a273caa13e9aa3e6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:34:58.518532 4666 log.cc:1196] T a15043ef2bc546dcbb1622d7013e5d09 P c96152fcc2c04079a273caa13e9aa3e6: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/ts-2/wal/wals/a15043ef2bc546dcbb1622d7013e5d09
I20260806 06:34:58.518810 4666 ts_tablet_manager.cc:1939] T a15043ef2bc546dcbb1622d7013e5d09 P c96152fcc2c04079a273caa13e9aa3e6: Deleting consensus metadata
I20260806 06:34:58.519255 3947 catalog_manager.cc:4841] TS c96152fcc2c04079a273caa13e9aa3e6 (127.7.43.67:37727): tablet a15043ef2bc546dcbb1622d7013e5d09 (table test-table [id=e5d38091b861416e96e6e7c8745c64e8]) successfully deleted
I20260806 06:34:58.519366 4667 ts_tablet_manager.cc:1905] T a15043ef2bc546dcbb1622d7013e5d09 P 0c95bf016a4644589a1a8f9c81aa397b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:34:58.520381 4667 ts_tablet_manager.cc:1918] T a15043ef2bc546dcbb1622d7013e5d09 P 0c95bf016a4644589a1a8f9c81aa397b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:34:58.520443 4667 log.cc:1196] T a15043ef2bc546dcbb1622d7013e5d09 P 0c95bf016a4644589a1a8f9c81aa397b: Deleting WAL directory at /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785997861828271-7341-0/minicluster-data/ts-1/wal/wals/a15043ef2bc546dcbb1622d7013e5d09
I20260806 06:34:58.520638 4667 ts_tablet_manager.cc:1939] T a15043ef2bc546dcbb1622d7013e5d09 P 0c95bf016a4644589a1a8f9c81aa397b: Deleting consensus metadata
I20260806 06:34:58.520977 3949 catalog_manager.cc:4841] TS 0c95bf016a4644589a1a8f9c81aa397b (127.7.43.66:39319): tablet a15043ef2bc546dcbb1622d7013e5d09 (table test-table [id=e5d38091b861416e96e6e7c8745c64e8]) successfully deleted
W20260806 06:34:58.526556 3949 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.43.65:42521 (error 108)
I20260806 06:34:58.526949 4622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:58.526845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.65:42521 (local address 127.0.0.1:48168)
0806 06:34:58.526889 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 06:34:58.526925 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.65:42521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 06:34:58.526981 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 4239
W20260806 06:34:58.527072 3949 catalog_manager.cc:4584] TS de4d73cb2c5049519be190014ea404a7 (127.7.43.65:42521): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a15043ef2bc546dcbb1622d7013e5d09: Network error: Client connection negotiation failed: client connection to 127.7.43.65:42521: connect: Connection refused (error 111)
I20260806 06:34:58.537518 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 4421
I20260806 06:34:58.544582 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 3924
2026-08-06T06:34:58Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (736 ms)
[----------] 6 tests from EncryptionPolicyTest (8308 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785997861828271-7341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:34:58 dist-test-slave-6zbq krb5kdc[4671](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:34:58 dist-test-slave-6zbq krb5kdc[4671](info): set up 2 sockets
Aug 06 06:34:58 dist-test-slave-6zbq krb5kdc[4671](info): commencing operation
krb5kdc: starting...
W20260806 06:34:59.647189 7341 mini_kdc.cc:121] Time spent starting KDC: real 1.081s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Aug 06 06:34:59 dist-test-slave-6zbq krb5kdc[4671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998099, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:34:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:34:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785997861828271-7341-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785997861828271-7341-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:34:59.754179 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45313
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785997861828271-7341-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45313
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:34:59.831533 4688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:34:59.831709 4688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:34:59.831737 4688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:34:59.832799 4688 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:34:59.832837 4688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:34:59.832855 4688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:34:59.832871 4688 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:34:59.832896 4688 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:34:59.834290 4688 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785997861828271-7341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45313
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785997861828271-7341-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45313
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4688
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:34:59.834537 4688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:34:59.834718 4688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:34:59.839094 4697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:34:59.839242 4688 server_base.cc:1034] running on GCE node
W20260806 06:34:59.839135 4701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:34:59.839093 4698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:34:59.839543 4688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:34:59.839784 4688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:34:59.840927 4688 hybrid_clock.cc:648] HybridClock initialized: now 1785998099840905 us; error 33 us; skew 500 ppm
Aug 06 06:34:59 dist-test-slave-6zbq krb5kdc[4671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998099, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:34:59.847572 4688 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:34:59.847985 4688 webserver.cc:460] Webserver started at http://127.0.0.1:41681/ using document root <none> and password file <none>
I20260806 06:34:59.848151 4688 fs_manager.cc:359] Metadata directory not provided
I20260806 06:34:59.848187 4688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:34:59.848271 4688 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:34:59.849179 4688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785997861828271-7341-0/minicluster-data/master-0/data/instance:
uuid: "da08a910febf4892b637f8e8896355b5"
format_stamp: "Formatted at 2026-08-06 06:34:59 on dist-test-slave-6zbq"
server_key: "3d9e074e3e90be0289f1555253c8778e"
server_key_iv: "1b7c7f2a0b6c5e1786d675119c44d376"
server_key_version: "encryptionkey@0"
I20260806 06:34:59.849455 4688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785997861828271-7341-0/minicluster-data/master-0/wal/instance:
uuid: "da08a910febf4892b637f8e8896355b5"
format_stamp: "Formatted at 2026-08-06 06:34:59 on dist-test-slave-6zbq"
server_key: "3d9e074e3e90be0289f1555253c8778e"
server_key_iv: "1b7c7f2a0b6c5e1786d675119c44d376"
server_key_version: "encryptionkey@0"
I20260806 06:34:59.850594 4688 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:34:59.851215 4708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:34:59.851357 4688 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:34:59.851437 4688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "da08a910febf4892b637f8e8896355b5"
format_stamp: "Formatted at 2026-08-06 06:34:59 on dist-test-slave-6zbq"
server_key: "3d9e074e3e90be0289f1555253c8778e"
server_key_iv: "1b7c7f2a0b6c5e1786d675119c44d376"
server_key_version: "encryptionkey@0"
I20260806 06:34:59.851495 4688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:34:59.888159 4688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:34:59.889724 4688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:34:59.889856 4688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:34:59.893527 4688 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45313
I20260806 06:34:59.893560 4784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45313 every 8 connection(s)
I20260806 06:34:59.893867 4688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:34:59.894779 4786 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:34:59.896909 4786 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da08a910febf4892b637f8e8896355b5: Bootstrap starting.
I20260806 06:34:59.897460 4786 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da08a910febf4892b637f8e8896355b5: Neither blocks nor log segments found. Creating new log.
I20260806 06:34:59.897706 4786 log.cc:824] T 00000000000000000000000000000000 P da08a910febf4892b637f8e8896355b5: Log is configured to *not* fsync() on all Append() calls
I20260806 06:34:59.898306 4786 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da08a910febf4892b637f8e8896355b5: No bootstrap required, opened a new log
I20260806 06:34:59.898298 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 4688
I20260806 06:34:59.898430 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785997861828271-7341-0/minicluster-data/master-0/wal/instance
I20260806 06:34:59.898608 7341 external_mini_cluster.cc:1428] Setting key 17b42d6414ba9428a3db7f7879e25da4
I20260806 06:34:59.900084 4786 raft_consensus.cc:348] T 00000000000000000000000000000000 P da08a910febf4892b637f8e8896355b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da08a910febf4892b637f8e8896355b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45313 } }
I20260806 06:34:59.900228 4786 raft_consensus.cc:374] T 00000000000000000000000000000000 P da08a910febf4892b637f8e8896355b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:34:59.900327 4786 raft_consensus.cc:729] T 00000000000000000000000000000000 P da08a910febf4892b637f8e8896355b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da08a910febf4892b637f8e8896355b5, State: Initialized, Role: FOLLOWER
I20260806 06:34:59.900465 4786 consensus_queue.cc:260] T 00000000000000000000000000000000 P da08a910febf4892b637f8e8896355b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da08a910febf4892b637f8e8896355b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45313 } }
I20260806 06:34:59.900545 4786 raft_consensus.cc:388] T 00000000000000000000000000000000 P da08a910febf4892b637f8e8896355b5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:34:59.900583 4786 raft_consensus.cc:482] T 00000000000000000000000000000000 P da08a910febf4892b637f8e8896355b5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:34:59.900629 4786 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da08a910febf4892b637f8e8896355b5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:34:59.901510 4786 raft_consensus.cc:504] T 00000000000000000000000000000000 P da08a910febf4892b637f8e8896355b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da08a910febf4892b637f8e8896355b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45313 } }
I20260806 06:34:59.901607 4786 leader_election.cc:302] T 00000000000000000000000000000000 P da08a910febf4892b637f8e8896355b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: da08a910febf4892b637f8e8896355b5; no voters:
I20260806 06:34:59.901750 4786 leader_election.cc:288] T 00000000000000000000000000000000 P da08a910febf4892b637f8e8896355b5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:34:59.901787 4794 raft_consensus.cc:2781] T 00000000000000000000000000000000 P da08a910febf4892b637f8e8896355b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:34:59.901927 4794 raft_consensus.cc:686] T 00000000000000000000000000000000 P da08a910febf4892b637f8e8896355b5 [term 1 LEADER]: Becoming Leader. State: Replica: da08a910febf4892b637f8e8896355b5, State: Running, Role: LEADER
I20260806 06:34:59.901993 4786 sys_catalog.cc:564] T 00000000000000000000000000000000 P da08a910febf4892b637f8e8896355b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:34:59.902031 4794 consensus_queue.cc:237] T 00000000000000000000000000000000 P da08a910febf4892b637f8e8896355b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da08a910febf4892b637f8e8896355b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45313 } }
I20260806 06:34:59.902360 4794 sys_catalog.cc:455] T 00000000000000000000000000000000 P da08a910febf4892b637f8e8896355b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader da08a910febf4892b637f8e8896355b5. Latest consensus state: current_term: 1 leader_uuid: "da08a910febf4892b637f8e8896355b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da08a910febf4892b637f8e8896355b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45313 } } }
I20260806 06:34:59.902593 4794 sys_catalog.cc:458] T 00000000000000000000000000000000 P da08a910febf4892b637f8e8896355b5 [sys.catalog]: This master's current role is: LEADER
I20260806 06:34:59.902525 4795 sys_catalog.cc:455] T 00000000000000000000000000000000 P da08a910febf4892b637f8e8896355b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "da08a910febf4892b637f8e8896355b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da08a910febf4892b637f8e8896355b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45313 } } }
I20260806 06:34:59.903008 4795 sys_catalog.cc:458] T 00000000000000000000000000000000 P da08a910febf4892b637f8e8896355b5 [sys.catalog]: This master's current role is: LEADER
I20260806 06:34:59.903028 4804 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:34:59.903566 4804 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 06:34:59 dist-test-slave-6zbq krb5kdc[4671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998099, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 06:34:59.905109 4804 catalog_manager.cc:1347] Generated new cluster ID: 5eeb7610cbb743339b31d81a998abd17
I20260806 06:34:59.905984 4804 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:34:59.909935 4793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:59.899057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49899 (local address 127.0.0.1:45313)
0806 06:34:59.899203 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:59.899210 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:59.899234 (+ 24us) server_negotiation.cc:407] Connection header received
0806 06:34:59.900055 (+ 821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:59.900061 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:59.900101 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:59.900202 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:34:59.900980 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:59.901958 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:59.903036 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:59.903586 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:59.904052 (+ 466us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:59.906376 (+ 2324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:34:59.906380 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:34:59.906380 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:34:59.906392 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:34:59.908327 (+ 1935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:34:59.908725 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:34:59.908728 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:34:59.908728 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:34:59.908867 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:34:59.909059 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:34:59.909059 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:34:59.909060 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:34:59.909350 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:34:59.909501 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:59.909604 (+ 103us) server_negotiation.cc:299] Negotiation successful
0806 06:34:59.909712 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":69,"threads_started":1}
I20260806 06:34:59.911139 7341 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:34:59.921181 4793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:34:59.914635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44741 (local address 127.0.0.1:45313)
0806 06:34:59.914668 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 06:34:59.914670 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:34:59.914802 (+ 132us) server_negotiation.cc:407] Connection header received
0806 06:34:59.914846 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:34:59.914847 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:34:59.914855 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:34:59.914892 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:34:59.915460 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:59.916146 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:59.917108 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:34:59.917393 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:34:59.917443 (+ 50us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:34:59.918698 (+ 1255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:34:59.918699 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:34:59.918699 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:34:59.918706 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:34:59.920110 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:34:59.920490 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:34:59.920491 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:34:59.920491 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:34:59.920538 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:34:59.920727 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:34:59.920728 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:34:59.920728 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:34:59.920832 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:34:59.920909 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 06:34:59.921036 (+ 127us) server_negotiation.cc:299] Negotiation successful
0806 06:34:59.921088 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:34:59.935786 4804 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:34:59.936228 4804 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:34:59.938521 4804 catalog_manager.cc:5868] T 00000000000000000000000000000000 P da08a910febf4892b637f8e8896355b5: Generated new TSK 0
I20260806 06:34:59.938661 4804 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:34:59.943029 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 4688
W20260806 06:34:59.951942 4669 connection.cc:535] client connection to 127.0.0.1:45313 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45313): Connection reset by peer (error 104)
2026-08-06T06:34:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1399 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785997861828271-7341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:34:59 dist-test-slave-6zbq krb5kdc[4823](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:34:59 dist-test-slave-6zbq krb5kdc[4823](info): set up 2 sockets
Aug 06 06:34:59 dist-test-slave-6zbq krb5kdc[4823](info): commencing operation
krb5kdc: starting...
W20260806 06:35:01.035986 7341 mini_kdc.cc:121] Time spent starting KDC: real 1.071s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Aug 06 06:35:01 dist-test-slave-6zbq krb5kdc[4823](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998101, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:35:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:35:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785997861828271-7341-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785997861828271-7341-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:35:01.144196 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36281
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785997861828271-7341-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36281
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:35:01.222370 4840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:35:01.222575 4840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:35:01.222606 4840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:35:01.223692 4840 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:35:01.223731 4840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:35:01.223758 4840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:35:01.223775 4840 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:35:01.223798 4840 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:35:01.225216 4840 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785997861828271-7341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36281
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785997861828271-7341-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:35:01.225440 4840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:35:01.225630 4840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:35:01.230029 4853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:01.230054 4849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:01.230085 4851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:01.230130 4840 server_base.cc:1034] running on GCE node
I20260806 06:35:01.230474 4840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:35:01.230727 4840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:35:01.231858 4840 hybrid_clock.cc:648] HybridClock initialized: now 1785998101231846 us; error 30 us; skew 500 ppm
Aug 06 06:35:01 dist-test-slave-6zbq krb5kdc[4823](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998101, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:35:01.234099 4840 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:35:01.234550 4840 webserver.cc:460] Webserver started at http://127.0.0.1:45505/ using document root <none> and password file <none>
I20260806 06:35:01.234737 4840 fs_manager.cc:359] Metadata directory not provided
I20260806 06:35:01.234781 4840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:35:01.234891 4840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:35:01.235841 4840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785997861828271-7341-0/minicluster-data/master-0/data/instance:
uuid: "11bd383337974cfe92ba7fd33b99a300"
format_stamp: "Formatted at 2026-08-06 06:35:01 on dist-test-slave-6zbq"
server_key: "7112d71732da653caa9f5b6a29de0703"
server_key_iv: "d12281f4e06d0b7ded9a9bab96789126"
server_key_version: "encryptionkey@0"
I20260806 06:35:01.236145 4840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785997861828271-7341-0/minicluster-data/master-0/wal/instance:
uuid: "11bd383337974cfe92ba7fd33b99a300"
format_stamp: "Formatted at 2026-08-06 06:35:01 on dist-test-slave-6zbq"
server_key: "7112d71732da653caa9f5b6a29de0703"
server_key_iv: "d12281f4e06d0b7ded9a9bab96789126"
server_key_version: "encryptionkey@0"
I20260806 06:35:01.237308 4840 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:35:01.237980 4864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:01.238148 4840 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:35:01.238233 4840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "11bd383337974cfe92ba7fd33b99a300"
format_stamp: "Formatted at 2026-08-06 06:35:01 on dist-test-slave-6zbq"
server_key: "7112d71732da653caa9f5b6a29de0703"
server_key_iv: "d12281f4e06d0b7ded9a9bab96789126"
server_key_version: "encryptionkey@0"
I20260806 06:35:01.238286 4840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:35:01.249392 4840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:35:01.251312 4840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:35:01.251412 4840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:35:01.254993 4840 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36281
I20260806 06:35:01.255023 4945 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36281 every 8 connection(s)
I20260806 06:35:01.255316 4840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:35:01.256168 4946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:35:01.258121 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 4840
I20260806 06:35:01.258256 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785997861828271-7341-0/minicluster-data/master-0/wal/instance
I20260806 06:35:01.258317 4946 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11bd383337974cfe92ba7fd33b99a300: Bootstrap starting.
I20260806 06:35:01.258493 7341 external_mini_cluster.cc:1428] Setting key 5b38fd3d18f04f1680b5714003f42d29
I20260806 06:35:01.258980 4946 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 11bd383337974cfe92ba7fd33b99a300: Neither blocks nor log segments found. Creating new log.
I20260806 06:35:01.259305 4946 log.cc:824] T 00000000000000000000000000000000 P 11bd383337974cfe92ba7fd33b99a300: Log is configured to *not* fsync() on all Append() calls
I20260806 06:35:01.259999 4946 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11bd383337974cfe92ba7fd33b99a300: No bootstrap required, opened a new log
I20260806 06:35:01.262153 4946 raft_consensus.cc:348] T 00000000000000000000000000000000 P 11bd383337974cfe92ba7fd33b99a300 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11bd383337974cfe92ba7fd33b99a300" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36281 } }
I20260806 06:35:01.262286 4946 raft_consensus.cc:374] T 00000000000000000000000000000000 P 11bd383337974cfe92ba7fd33b99a300 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:35:01.262328 4946 raft_consensus.cc:729] T 00000000000000000000000000000000 P 11bd383337974cfe92ba7fd33b99a300 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11bd383337974cfe92ba7fd33b99a300, State: Initialized, Role: FOLLOWER
I20260806 06:35:01.262472 4946 consensus_queue.cc:260] T 00000000000000000000000000000000 P 11bd383337974cfe92ba7fd33b99a300 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11bd383337974cfe92ba7fd33b99a300" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36281 } }
I20260806 06:35:01.262545 4946 raft_consensus.cc:388] T 00000000000000000000000000000000 P 11bd383337974cfe92ba7fd33b99a300 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:35:01.262581 4946 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11bd383337974cfe92ba7fd33b99a300 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:35:01.262624 4946 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 11bd383337974cfe92ba7fd33b99a300 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:35:01.263252 4946 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11bd383337974cfe92ba7fd33b99a300 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11bd383337974cfe92ba7fd33b99a300" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36281 } }
I20260806 06:35:01.263371 4946 leader_election.cc:302] T 00000000000000000000000000000000 P 11bd383337974cfe92ba7fd33b99a300 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 11bd383337974cfe92ba7fd33b99a300; no voters:
I20260806 06:35:01.263527 4946 leader_election.cc:288] T 00000000000000000000000000000000 P 11bd383337974cfe92ba7fd33b99a300 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:35:01.263585 4953 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 11bd383337974cfe92ba7fd33b99a300 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:35:01.263706 4953 raft_consensus.cc:686] T 00000000000000000000000000000000 P 11bd383337974cfe92ba7fd33b99a300 [term 1 LEADER]: Becoming Leader. State: Replica: 11bd383337974cfe92ba7fd33b99a300, State: Running, Role: LEADER
I20260806 06:35:01.263790 4953 consensus_queue.cc:237] T 00000000000000000000000000000000 P 11bd383337974cfe92ba7fd33b99a300 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11bd383337974cfe92ba7fd33b99a300" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36281 } }
I20260806 06:35:01.263841 4946 sys_catalog.cc:564] T 00000000000000000000000000000000 P 11bd383337974cfe92ba7fd33b99a300 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:35:01.264088 4953 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11bd383337974cfe92ba7fd33b99a300 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "11bd383337974cfe92ba7fd33b99a300" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11bd383337974cfe92ba7fd33b99a300" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36281 } } }
I20260806 06:35:01.264204 4953 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11bd383337974cfe92ba7fd33b99a300 [sys.catalog]: This master's current role is: LEADER
I20260806 06:35:01.264343 4951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:01.258958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36795 (local address 127.0.0.1:36281)
0806 06:35:01.259188 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:01.259197 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:01.259226 (+ 29us) server_negotiation.cc:407] Connection header received
0806 06:35:01.259737 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:01.259746 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:01.259807 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:01.259941 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:01.260160 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:01.261490 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:01.262287 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:01.262873 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:01.262983 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:01.263604 (+ 621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:01.263608 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:01.263608 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:35:01.263812 (+ 204us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:01.263849 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:35:01.263864 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:01.263946 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:01.264044 (+ 98us) server_negotiation.cc:299] Negotiation successful
0806 06:35:01.264070 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":65,"threads_started":1}
I20260806 06:35:01.264351 4953 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11bd383337974cfe92ba7fd33b99a300 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11bd383337974cfe92ba7fd33b99a300. Latest consensus state: current_term: 1 leader_uuid: "11bd383337974cfe92ba7fd33b99a300" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11bd383337974cfe92ba7fd33b99a300" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36281 } } }
I20260806 06:35:01.264612 4953 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11bd383337974cfe92ba7fd33b99a300 [sys.catalog]: This master's current role is: LEADER
I20260806 06:35:01.264436 4960 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:35:01.265524 7341 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:35:01.265640 4960 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:35:01.267149 4960 catalog_manager.cc:1347] Generated new cluster ID: 1a18a31a99274d059572b8e772b4c2c2
I20260806 06:35:01.267199 4960 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:35:01.273870 4951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:01.266948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36623 (local address 127.0.0.1:36281)
0806 06:35:01.266986 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:01.266988 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:01.270386 (+ 3398us) server_negotiation.cc:407] Connection header received
0806 06:35:01.270493 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:01.270494 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:01.270505 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:01.270562 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:01.270689 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:01.271722 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:01.272656 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:01.272983 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:01.273054 (+ 71us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:01.273640 (+ 586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:01.273641 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:01.273642 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:35:01.273646 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:01.273661 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:35:01.273666 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:01.273721 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:01.273789 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 06:35:01.273799 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:35:01.277217 4960 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:35:01.277678 4960 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:35:01.284442 4960 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 11bd383337974cfe92ba7fd33b99a300: Generated new TSK 0
I20260806 06:35:01.284591 4960 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:35:01.287446 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 4840
W20260806 06:35:01.299675 4821 connection.cc:535] client connection to 127.0.0.1:36281 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36281): Connection reset by peer (error 104)
2026-08-06T06:35:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1347 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785997861828271-7341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:35:01 dist-test-slave-6zbq krb5kdc[4993](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:35:01 dist-test-slave-6zbq krb5kdc[4993](info): set up 2 sockets
Aug 06 06:35:01 dist-test-slave-6zbq krb5kdc[4993](info): commencing operation
krb5kdc: starting...
W20260806 06:35:02.404831 7341 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Aug 06 06:35:02 dist-test-slave-6zbq krb5kdc[4993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998102, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:35:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:35:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785997861828271-7341-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785997861828271-7341-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:35:02.511092 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38575
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785997861828271-7341-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38575
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:35:02.587980 5006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:35:02.588155 5006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:35:02.588183 5006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:35:02.589265 5006 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:35:02.589304 5006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:35:02.589322 5006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:35:02.589339 5006 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:35:02.589354 5006 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:35:02.590787 5006 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785997861828271-7341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38575
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785997861828271-7341-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5006
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:35:02.591017 5006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:35:02.591200 5006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:35:02.595660 5013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:02.595697 5006 server_base.cc:1034] running on GCE node
W20260806 06:35:02.595602 5018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:02.595604 5014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:02.596076 5006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:35:02.596343 5006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:35:02.597473 5006 hybrid_clock.cc:648] HybridClock initialized: now 1785998102597462 us; error 28 us; skew 500 ppm
Aug 06 06:35:02 dist-test-slave-6zbq krb5kdc[4993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998102, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:35:02.599687 5006 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:35:02.600056 5006 webserver.cc:460] Webserver started at http://127.0.0.1:40961/ using document root <none> and password file <none>
I20260806 06:35:02.600238 5006 fs_manager.cc:359] Metadata directory not provided
I20260806 06:35:02.600291 5006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:35:02.600407 5006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:35:02.601617 5006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785997861828271-7341-0/minicluster-data/master-0/data/instance:
uuid: "e03befe267064a0e95e0d54f41b74985"
format_stamp: "Formatted at 2026-08-06 06:35:02 on dist-test-slave-6zbq"
server_key: "7f37cc214fc17756bcf6f6bcc65a3b27"
server_key_iv: "ce3dc903e2d02904948ae5bcb8159e4e"
server_key_version: "encryptionkey@0"
I20260806 06:35:02.601939 5006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785997861828271-7341-0/minicluster-data/master-0/wal/instance:
uuid: "e03befe267064a0e95e0d54f41b74985"
format_stamp: "Formatted at 2026-08-06 06:35:02 on dist-test-slave-6zbq"
server_key: "7f37cc214fc17756bcf6f6bcc65a3b27"
server_key_iv: "ce3dc903e2d02904948ae5bcb8159e4e"
server_key_version: "encryptionkey@0"
I20260806 06:35:02.603163 5006 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:35:02.604019 5028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:02.604176 5006 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:35:02.604256 5006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "e03befe267064a0e95e0d54f41b74985"
format_stamp: "Formatted at 2026-08-06 06:35:02 on dist-test-slave-6zbq"
server_key: "7f37cc214fc17756bcf6f6bcc65a3b27"
server_key_iv: "ce3dc903e2d02904948ae5bcb8159e4e"
server_key_version: "encryptionkey@0"
I20260806 06:35:02.604327 5006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:35:02.608592 5006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:35:02.610998 5006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:35:02.611166 5006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:35:02.614815 5006 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38575
I20260806 06:35:02.614845 5102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38575 every 8 connection(s)
I20260806 06:35:02.615119 5006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:35:02.616007 5104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:35:02.618028 5104 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e03befe267064a0e95e0d54f41b74985: Bootstrap starting.
I20260806 06:35:02.618573 5104 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e03befe267064a0e95e0d54f41b74985: Neither blocks nor log segments found. Creating new log.
I20260806 06:35:02.618813 5104 log.cc:824] T 00000000000000000000000000000000 P e03befe267064a0e95e0d54f41b74985: Log is configured to *not* fsync() on all Append() calls
I20260806 06:35:02.619475 5104 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e03befe267064a0e95e0d54f41b74985: No bootstrap required, opened a new log
I20260806 06:35:02.620999 5104 raft_consensus.cc:348] T 00000000000000000000000000000000 P e03befe267064a0e95e0d54f41b74985 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e03befe267064a0e95e0d54f41b74985" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38575 } }
I20260806 06:35:02.621150 5104 raft_consensus.cc:374] T 00000000000000000000000000000000 P e03befe267064a0e95e0d54f41b74985 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:35:02.621189 5104 raft_consensus.cc:729] T 00000000000000000000000000000000 P e03befe267064a0e95e0d54f41b74985 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e03befe267064a0e95e0d54f41b74985, State: Initialized, Role: FOLLOWER
I20260806 06:35:02.621320 5104 consensus_queue.cc:260] T 00000000000000000000000000000000 P e03befe267064a0e95e0d54f41b74985 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e03befe267064a0e95e0d54f41b74985" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38575 } }
I20260806 06:35:02.621380 5104 raft_consensus.cc:388] T 00000000000000000000000000000000 P e03befe267064a0e95e0d54f41b74985 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:35:02.621413 5104 raft_consensus.cc:482] T 00000000000000000000000000000000 P e03befe267064a0e95e0d54f41b74985 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:35:02.621457 5104 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e03befe267064a0e95e0d54f41b74985 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:35:02.621937 5104 raft_consensus.cc:504] T 00000000000000000000000000000000 P e03befe267064a0e95e0d54f41b74985 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e03befe267064a0e95e0d54f41b74985" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38575 } }
I20260806 06:35:02.622032 5104 leader_election.cc:302] T 00000000000000000000000000000000 P e03befe267064a0e95e0d54f41b74985 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e03befe267064a0e95e0d54f41b74985; no voters:
I20260806 06:35:02.622197 5104 leader_election.cc:288] T 00000000000000000000000000000000 P e03befe267064a0e95e0d54f41b74985 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:35:02.622224 5109 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e03befe267064a0e95e0d54f41b74985 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:35:02.622362 5109 raft_consensus.cc:686] T 00000000000000000000000000000000 P e03befe267064a0e95e0d54f41b74985 [term 1 LEADER]: Becoming Leader. State: Replica: e03befe267064a0e95e0d54f41b74985, State: Running, Role: LEADER
I20260806 06:35:02.622449 5104 sys_catalog.cc:564] T 00000000000000000000000000000000 P e03befe267064a0e95e0d54f41b74985 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:35:02.622493 5109 consensus_queue.cc:237] T 00000000000000000000000000000000 P e03befe267064a0e95e0d54f41b74985 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e03befe267064a0e95e0d54f41b74985" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38575 } }
I20260806 06:35:02.622853 5112 sys_catalog.cc:455] T 00000000000000000000000000000000 P e03befe267064a0e95e0d54f41b74985 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e03befe267064a0e95e0d54f41b74985. Latest consensus state: current_term: 1 leader_uuid: "e03befe267064a0e95e0d54f41b74985" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e03befe267064a0e95e0d54f41b74985" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38575 } } }
I20260806 06:35:02.622952 5110 sys_catalog.cc:455] T 00000000000000000000000000000000 P e03befe267064a0e95e0d54f41b74985 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e03befe267064a0e95e0d54f41b74985" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e03befe267064a0e95e0d54f41b74985" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38575 } } }
I20260806 06:35:02.622983 5112 sys_catalog.cc:458] T 00000000000000000000000000000000 P e03befe267064a0e95e0d54f41b74985 [sys.catalog]: This master's current role is: LEADER
I20260806 06:35:02.623037 5110 sys_catalog.cc:458] T 00000000000000000000000000000000 P e03befe267064a0e95e0d54f41b74985 [sys.catalog]: This master's current role is: LEADER
I20260806 06:35:02.623399 5120 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:35:02.623822 5120 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:35:02.624722 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 5006
I20260806 06:35:02.624853 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785997861828271-7341-0/minicluster-data/master-0/wal/instance
I20260806 06:35:02.625020 7341 external_mini_cluster.cc:1428] Setting key 551de60b65eb5d7c96dcdc96ec70110d
I20260806 06:35:02.625641 5120 catalog_manager.cc:1347] Generated new cluster ID: 7f743a29c8fd4a5ba24a4d52e636ba80
I20260806 06:35:02.625705 5120 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:35:02.627033 5133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:02.625559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47513 (local address 127.0.0.1:38575)
0806 06:35:02.625704 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:02.625712 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:02.625741 (+ 29us) server_negotiation.cc:407] Connection header received
0806 06:35:02.625842 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:02.625847 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:02.625870 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:02.625974 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:02.626650 (+ 676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:02.626654 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:02.626654 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:35:02.626658 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:02.626695 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:35:02.626705 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:02.626755 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:02.626824 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 06:35:02.626864 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":70,"threads_started":1}
I20260806 06:35:02.627755 7341 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:35:02.629832 5133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:02.628933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52583 (local address 127.0.0.1:38575)
0806 06:35:02.629023 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:02.629024 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:02.629076 (+ 52us) server_negotiation.cc:407] Connection header received
0806 06:35:02.629130 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:02.629131 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:02.629136 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:02.629179 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:02.629640 (+ 461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:02.629641 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:02.629641 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:35:02.629645 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:02.629658 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:35:02.629662 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:02.629707 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:02.629754 (+ 47us) server_negotiation.cc:299] Negotiation successful
0806 06:35:02.629767 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 06:35:02.634068 5120 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:35:02.634208 5120 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:35:02.639580 5120 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e03befe267064a0e95e0d54f41b74985: Generated new TSK 0
I20260806 06:35:02.639732 5120 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:35:02.647292 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 5006
2026-08-06T06:35:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1359 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785997861828271-7341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:35:02 dist-test-slave-6zbq krb5kdc[5141](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:35:02 dist-test-slave-6zbq krb5kdc[5141](info): set up 2 sockets
Aug 06 06:35:02 dist-test-slave-6zbq krb5kdc[5141](info): commencing operation
krb5kdc: starting...
W20260806 06:35:03.750393 7341 mini_kdc.cc:121] Time spent starting KDC: real 1.078s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Aug 06 06:35:03 dist-test-slave-6zbq krb5kdc[5141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998103, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:35:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:35:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785997861828271-7341-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785997861828271-7341-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:35:03.857127 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46371
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785997861828271-7341-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46371
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:35:03.934641 5163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:35:03.934827 5163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:35:03.934859 5163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:35:03.935945 5163 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:35:03.935984 5163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:35:03.936003 5163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:35:03.936019 5163 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:35:03.936043 5163 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:35:03.937439 5163 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785997861828271-7341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46371
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785997861828271-7341-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46371
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5163
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:35:03.937659 5163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:35:03.937845 5163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:35:03.942144 5173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:03.942162 5171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:03.942152 5176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:03.942306 5163 server_base.cc:1034] running on GCE node
I20260806 06:35:03.942567 5163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:35:03.942831 5163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:35:03.943962 5163 hybrid_clock.cc:648] HybridClock initialized: now 1785998103943949 us; error 29 us; skew 500 ppm
Aug 06 06:35:03 dist-test-slave-6zbq krb5kdc[5141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998103, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:35:03.946201 5163 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:35:03.946609 5163 webserver.cc:460] Webserver started at http://127.0.0.1:37727/ using document root <none> and password file <none>
I20260806 06:35:03.946777 5163 fs_manager.cc:359] Metadata directory not provided
I20260806 06:35:03.946822 5163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:35:03.946908 5163 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:35:03.947824 5163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785997861828271-7341-0/minicluster-data/master-0/data/instance:
uuid: "845ebddf4ae14e2ca14fc6fdf5b90bc6"
format_stamp: "Formatted at 2026-08-06 06:35:03 on dist-test-slave-6zbq"
server_key: "b7c8d0bbb1e8ad5f069770dc8f5d0549"
server_key_iv: "3f976cc6aa9796f1407fbee5f59b061e"
server_key_version: "encryptionkey@0"
I20260806 06:35:03.948100 5163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785997861828271-7341-0/minicluster-data/master-0/wal/instance:
uuid: "845ebddf4ae14e2ca14fc6fdf5b90bc6"
format_stamp: "Formatted at 2026-08-06 06:35:03 on dist-test-slave-6zbq"
server_key: "b7c8d0bbb1e8ad5f069770dc8f5d0549"
server_key_iv: "3f976cc6aa9796f1407fbee5f59b061e"
server_key_version: "encryptionkey@0"
I20260806 06:35:03.949201 5163 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:35:03.949780 5187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:03.949920 5163 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:35:03.950001 5163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "845ebddf4ae14e2ca14fc6fdf5b90bc6"
format_stamp: "Formatted at 2026-08-06 06:35:03 on dist-test-slave-6zbq"
server_key: "b7c8d0bbb1e8ad5f069770dc8f5d0549"
server_key_iv: "3f976cc6aa9796f1407fbee5f59b061e"
server_key_version: "encryptionkey@0"
I20260806 06:35:03.950057 5163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:35:04.009527 5163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:35:04.011513 5163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:35:04.011641 5163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:35:04.015410 5163 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46371
I20260806 06:35:04.015450 5266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46371 every 8 connection(s)
I20260806 06:35:04.015758 5163 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:35:04.016649 5268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:35:04.018872 5268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 845ebddf4ae14e2ca14fc6fdf5b90bc6: Bootstrap starting.
I20260806 06:35:04.019380 5268 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 845ebddf4ae14e2ca14fc6fdf5b90bc6: Neither blocks nor log segments found. Creating new log.
I20260806 06:35:04.019601 5268 log.cc:824] T 00000000000000000000000000000000 P 845ebddf4ae14e2ca14fc6fdf5b90bc6: Log is configured to *not* fsync() on all Append() calls
I20260806 06:35:04.020201 5268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 845ebddf4ae14e2ca14fc6fdf5b90bc6: No bootstrap required, opened a new log
I20260806 06:35:04.021341 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 5163
I20260806 06:35:04.021469 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785997861828271-7341-0/minicluster-data/master-0/wal/instance
I20260806 06:35:04.021646 7341 external_mini_cluster.cc:1428] Setting key 9de2fa919bc287752cbd5af6a5772f63
I20260806 06:35:04.022001 5268 raft_consensus.cc:348] T 00000000000000000000000000000000 P 845ebddf4ae14e2ca14fc6fdf5b90bc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "845ebddf4ae14e2ca14fc6fdf5b90bc6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46371 } }
I20260806 06:35:04.022174 5268 raft_consensus.cc:374] T 00000000000000000000000000000000 P 845ebddf4ae14e2ca14fc6fdf5b90bc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:35:04.022284 5268 raft_consensus.cc:729] T 00000000000000000000000000000000 P 845ebddf4ae14e2ca14fc6fdf5b90bc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 845ebddf4ae14e2ca14fc6fdf5b90bc6, State: Initialized, Role: FOLLOWER
I20260806 06:35:04.022446 5268 consensus_queue.cc:260] T 00000000000000000000000000000000 P 845ebddf4ae14e2ca14fc6fdf5b90bc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "845ebddf4ae14e2ca14fc6fdf5b90bc6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46371 } }
I20260806 06:35:04.022526 5268 raft_consensus.cc:388] T 00000000000000000000000000000000 P 845ebddf4ae14e2ca14fc6fdf5b90bc6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:35:04.022557 5268 raft_consensus.cc:482] T 00000000000000000000000000000000 P 845ebddf4ae14e2ca14fc6fdf5b90bc6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:35:04.022640 5268 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 845ebddf4ae14e2ca14fc6fdf5b90bc6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:35:04.023600 5268 raft_consensus.cc:504] T 00000000000000000000000000000000 P 845ebddf4ae14e2ca14fc6fdf5b90bc6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "845ebddf4ae14e2ca14fc6fdf5b90bc6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46371 } }
I20260806 06:35:04.023698 5268 leader_election.cc:302] T 00000000000000000000000000000000 P 845ebddf4ae14e2ca14fc6fdf5b90bc6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 845ebddf4ae14e2ca14fc6fdf5b90bc6; no voters:
I20260806 06:35:04.023845 5268 leader_election.cc:288] T 00000000000000000000000000000000 P 845ebddf4ae14e2ca14fc6fdf5b90bc6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:35:04.023878 5277 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 845ebddf4ae14e2ca14fc6fdf5b90bc6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:35:04.024020 5277 raft_consensus.cc:686] T 00000000000000000000000000000000 P 845ebddf4ae14e2ca14fc6fdf5b90bc6 [term 1 LEADER]: Becoming Leader. State: Replica: 845ebddf4ae14e2ca14fc6fdf5b90bc6, State: Running, Role: LEADER
I20260806 06:35:04.024097 5268 sys_catalog.cc:564] T 00000000000000000000000000000000 P 845ebddf4ae14e2ca14fc6fdf5b90bc6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:35:04.024137 5277 consensus_queue.cc:237] T 00000000000000000000000000000000 P 845ebddf4ae14e2ca14fc6fdf5b90bc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "845ebddf4ae14e2ca14fc6fdf5b90bc6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46371 } }
I20260806 06:35:04.024549 5277 sys_catalog.cc:455] T 00000000000000000000000000000000 P 845ebddf4ae14e2ca14fc6fdf5b90bc6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 845ebddf4ae14e2ca14fc6fdf5b90bc6. Latest consensus state: current_term: 1 leader_uuid: "845ebddf4ae14e2ca14fc6fdf5b90bc6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "845ebddf4ae14e2ca14fc6fdf5b90bc6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46371 } } }
I20260806 06:35:04.024605 5280 sys_catalog.cc:455] T 00000000000000000000000000000000 P 845ebddf4ae14e2ca14fc6fdf5b90bc6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "845ebddf4ae14e2ca14fc6fdf5b90bc6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "845ebddf4ae14e2ca14fc6fdf5b90bc6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46371 } } }
I20260806 06:35:04.024701 5280 sys_catalog.cc:458] T 00000000000000000000000000000000 P 845ebddf4ae14e2ca14fc6fdf5b90bc6 [sys.catalog]: This master's current role is: LEADER
I20260806 06:35:04.024677 5277 sys_catalog.cc:458] T 00000000000000000000000000000000 P 845ebddf4ae14e2ca14fc6fdf5b90bc6 [sys.catalog]: This master's current role is: LEADER
I20260806 06:35:04.025054 5295 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:35:04.025575 5295 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:35:04.027127 5295 catalog_manager.cc:1347] Generated new cluster ID: 2613e07e77c04ece8fbc4c20d7ad8f09
I20260806 06:35:04.027179 5295 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:35:04 dist-test-slave-6zbq krb5kdc[5141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998103, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 06:35:04.032362 5275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:04.022124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56071 (local address 127.0.0.1:46371)
0806 06:35:04.022295 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:04.022304 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:04.022328 (+ 24us) server_negotiation.cc:407] Connection header received
0806 06:35:04.022456 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:04.022462 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:04.022502 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:04.022622 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:04.023348 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:04.024450 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:04.025489 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:04.026116 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:04.026245 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:04.028794 (+ 2549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:04.028798 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:04.028798 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:04.028809 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:04.030614 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:04.030954 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:04.030957 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:04.030957 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:04.031066 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:04.031238 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:04.031239 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:04.031239 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:04.031512 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:04.031642 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:04.032027 (+ 385us) server_negotiation.cc:299] Negotiation successful
0806 06:35:04.032152 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":78,"threads_started":1}
I20260806 06:35:04.033190 5295 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:35:04.033243 7341 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:35:04.033844 5295 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:35:04.038492 5295 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 845ebddf4ae14e2ca14fc6fdf5b90bc6: Generated new TSK 0
I20260806 06:35:04.038638 5295 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:35:04.040748 5275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:04.034624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42725 (local address 127.0.0.1:46371)
0806 06:35:04.034660 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:04.034662 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:04.034760 (+ 98us) server_negotiation.cc:407] Connection header received
0806 06:35:04.034816 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:04.034816 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:04.034825 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:04.034876 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:04.035443 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:04.036125 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:04.037138 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:04.037426 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:04.037474 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:04.038727 (+ 1253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:04.038728 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:04.038728 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:04.038736 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:04.039863 (+ 1127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:04.040215 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:04.040215 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:04.040216 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:04.040255 (+ 39us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:04.040402 (+ 147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:04.040402 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:04.040402 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:04.040478 (+ 76us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:04.040535 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:04.040649 (+ 114us) server_negotiation.cc:299] Negotiation successful
0806 06:35:04.040698 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260806 06:35:04.042313 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 5163
2026-08-06T06:35:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1394 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785997861828271-7341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:35:04 dist-test-slave-6zbq krb5kdc[5326](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:35:04 dist-test-slave-6zbq krb5kdc[5326](info): set up 2 sockets
Aug 06 06:35:04 dist-test-slave-6zbq krb5kdc[5326](info): commencing operation
krb5kdc: starting...
W20260806 06:35:05.148164 7341 mini_kdc.cc:121] Time spent starting KDC: real 1.081s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Aug 06 06:35:05 dist-test-slave-6zbq krb5kdc[5326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998105, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:35:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:35:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785997861828271-7341-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785997861828271-7341-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:35:05.255266 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33865
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785997861828271-7341-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33865
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:35:05.334002 5347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:35:05.334190 5347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:35:05.334221 5347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:35:05.335328 5347 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:35:05.335368 5347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:35:05.335386 5347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:35:05.335403 5347 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:35:05.335418 5347 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:35:05.336828 5347 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785997861828271-7341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33865
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785997861828271-7341-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33865
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5347
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:35:05.337054 5347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:35:05.337252 5347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:35:05.341696 5356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:05.341686 5358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:05.341684 5355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:05.341892 5347 server_base.cc:1034] running on GCE node
I20260806 06:35:05.342136 5347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:35:05.342434 5347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:35:05.343569 5347 hybrid_clock.cc:648] HybridClock initialized: now 1785998105343558 us; error 27 us; skew 500 ppm
Aug 06 06:35:05 dist-test-slave-6zbq krb5kdc[5326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998105, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:35:05.351133 5347 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:35:05.351523 5347 webserver.cc:460] Webserver started at http://127.0.0.1:32773/ using document root <none> and password file <none>
I20260806 06:35:05.351732 5347 fs_manager.cc:359] Metadata directory not provided
I20260806 06:35:05.351778 5347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:35:05.351891 5347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:35:05.352864 5347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785997861828271-7341-0/minicluster-data/master-0/data/instance:
uuid: "2cac9f875436489dad7f8afa17aa5e54"
format_stamp: "Formatted at 2026-08-06 06:35:05 on dist-test-slave-6zbq"
server_key: "68b88e2257b7ce58dbd24c3961884bbb"
server_key_iv: "b7188ed8fcf3042f935f65308278aff2"
server_key_version: "encryptionkey@0"
I20260806 06:35:05.353176 5347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785997861828271-7341-0/minicluster-data/master-0/wal/instance:
uuid: "2cac9f875436489dad7f8afa17aa5e54"
format_stamp: "Formatted at 2026-08-06 06:35:05 on dist-test-slave-6zbq"
server_key: "68b88e2257b7ce58dbd24c3961884bbb"
server_key_iv: "b7188ed8fcf3042f935f65308278aff2"
server_key_version: "encryptionkey@0"
I20260806 06:35:05.354399 5347 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:35:05.355099 5368 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:05.355250 5347 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:05.355345 5347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "2cac9f875436489dad7f8afa17aa5e54"
format_stamp: "Formatted at 2026-08-06 06:35:05 on dist-test-slave-6zbq"
server_key: "68b88e2257b7ce58dbd24c3961884bbb"
server_key_iv: "b7188ed8fcf3042f935f65308278aff2"
server_key_version: "encryptionkey@0"
I20260806 06:35:05.355407 5347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:35:05.371423 5347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:35:05.373013 5347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:35:05.373135 5347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:35:05.376724 5347 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33865
I20260806 06:35:05.376762 5450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33865 every 8 connection(s)
I20260806 06:35:05.377053 5347 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:35:05.377925 5451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:35:05.379431 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 5347
I20260806 06:35:05.379556 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785997861828271-7341-0/minicluster-data/master-0/wal/instance
I20260806 06:35:05.379729 7341 external_mini_cluster.cc:1428] Setting key 4292a4087d9de472f1f866134ba26191
I20260806 06:35:05.380396 5451 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cac9f875436489dad7f8afa17aa5e54: Bootstrap starting.
I20260806 06:35:05.380947 5451 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2cac9f875436489dad7f8afa17aa5e54: Neither blocks nor log segments found. Creating new log.
I20260806 06:35:05.381276 5451 log.cc:824] T 00000000000000000000000000000000 P 2cac9f875436489dad7f8afa17aa5e54: Log is configured to *not* fsync() on all Append() calls
I20260806 06:35:05.381877 5451 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cac9f875436489dad7f8afa17aa5e54: No bootstrap required, opened a new log
I20260806 06:35:05.384052 5451 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2cac9f875436489dad7f8afa17aa5e54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cac9f875436489dad7f8afa17aa5e54" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33865 } }
I20260806 06:35:05.384222 5451 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2cac9f875436489dad7f8afa17aa5e54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:35:05.384255 5451 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2cac9f875436489dad7f8afa17aa5e54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cac9f875436489dad7f8afa17aa5e54, State: Initialized, Role: FOLLOWER
I20260806 06:35:05.384375 5451 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2cac9f875436489dad7f8afa17aa5e54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cac9f875436489dad7f8afa17aa5e54" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33865 } }
I20260806 06:35:05.384447 5451 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2cac9f875436489dad7f8afa17aa5e54 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:35:05.384471 5451 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2cac9f875436489dad7f8afa17aa5e54 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:35:05.384547 5451 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2cac9f875436489dad7f8afa17aa5e54 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:35:05.384788 5455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:05.380193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56459 (local address 127.0.0.1:33865)
0806 06:35:05.380353 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:05.380360 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:05.380384 (+ 24us) server_negotiation.cc:407] Connection header received
0806 06:35:05.380813 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:05.380819 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:05.380865 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:05.380972 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:05.381201 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:05.382095 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:05.383165 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:05.383541 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:05.383621 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:05.384197 (+ 576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:05.384201 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:05.384201 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:35:05.384389 (+ 188us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:05.384418 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:35:05.384429 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:05.384488 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:05.384572 (+ 84us) server_negotiation.cc:299] Negotiation successful
0806 06:35:05.384633 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":65,"threads_started":1}
I20260806 06:35:05.385159 5451 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2cac9f875436489dad7f8afa17aa5e54 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cac9f875436489dad7f8afa17aa5e54" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33865 } }
I20260806 06:35:05.385259 5451 leader_election.cc:302] T 00000000000000000000000000000000 P 2cac9f875436489dad7f8afa17aa5e54 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2cac9f875436489dad7f8afa17aa5e54; no voters:
I20260806 06:35:05.385416 5451 leader_election.cc:288] T 00000000000000000000000000000000 P 2cac9f875436489dad7f8afa17aa5e54 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:35:05.385448 5458 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2cac9f875436489dad7f8afa17aa5e54 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:35:05.385599 5458 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2cac9f875436489dad7f8afa17aa5e54 [term 1 LEADER]: Becoming Leader. State: Replica: 2cac9f875436489dad7f8afa17aa5e54, State: Running, Role: LEADER
I20260806 06:35:05.385636 7341 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:35:05.385649 5451 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2cac9f875436489dad7f8afa17aa5e54 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:35:05.385720 5458 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2cac9f875436489dad7f8afa17aa5e54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cac9f875436489dad7f8afa17aa5e54" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33865 } }
I20260806 06:35:05.386044 5458 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cac9f875436489dad7f8afa17aa5e54 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2cac9f875436489dad7f8afa17aa5e54. Latest consensus state: current_term: 1 leader_uuid: "2cac9f875436489dad7f8afa17aa5e54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cac9f875436489dad7f8afa17aa5e54" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33865 } } }
I20260806 06:35:05.386106 5458 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cac9f875436489dad7f8afa17aa5e54 [sys.catalog]: This master's current role is: LEADER
I20260806 06:35:05.386093 5464 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cac9f875436489dad7f8afa17aa5e54 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2cac9f875436489dad7f8afa17aa5e54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cac9f875436489dad7f8afa17aa5e54" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33865 } } }
I20260806 06:35:05.386170 5464 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cac9f875436489dad7f8afa17aa5e54 [sys.catalog]: This master's current role is: LEADER
I20260806 06:35:05.386310 5471 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:35:05.386945 5471 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:35:05.388540 5471 catalog_manager.cc:1347] Generated new cluster ID: 48cfdd7d0cb049598bfeeede671e42a7
I20260806 06:35:05.388587 5471 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:35:05.390524 5455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:05.387189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35147 (local address 127.0.0.1:33865)
0806 06:35:05.387222 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:05.387223 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:05.387492 (+ 269us) server_negotiation.cc:407] Connection header received
0806 06:35:05.387537 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:05.387538 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:05.387549 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:05.387594 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:05.387720 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:05.388253 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:05.389211 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:05.389510 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:05.389567 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:05.390195 (+ 628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:05.390196 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:05.390196 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:35:05.390200 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:05.390213 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:35:05.390217 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:05.390269 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:05.390433 (+ 164us) server_negotiation.cc:299] Negotiation successful
0806 06:35:05.390454 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:35:05.401865 5471 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:35:05.402598 5471 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:35:05.407485 5471 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2cac9f875436489dad7f8afa17aa5e54: Generated new TSK 0
I20260806 06:35:05.407624 5471 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:35:05.412060 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 5347
2026-08-06T06:35:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1373 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785997861828271-7341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:35:05 dist-test-slave-6zbq krb5kdc[5503](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:35:05 dist-test-slave-6zbq krb5kdc[5503](info): set up 2 sockets
Aug 06 06:35:05 dist-test-slave-6zbq krb5kdc[5503](info): commencing operation
krb5kdc: starting...
W20260806 06:35:06.527956 7341 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Aug 06 06:35:06 dist-test-slave-6zbq krb5kdc[5503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:35:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:35:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785997861828271-7341-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785997861828271-7341-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:35:06.646940 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46527
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785997861828271-7341-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46527
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:35:06.727541 5521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:35:06.727716 5521 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:35:06.727737 5521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:35:06.728832 5521 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:35:06.728871 5521 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:35:06.728884 5521 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:35:06.728897 5521 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:35:06.728921 5521 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:35:06.730324 5521 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785997861828271-7341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46527
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785997861828271-7341-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5521
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:35:06.730558 5521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:35:06.730723 5521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:35:06.735054 5527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:06.735025 5533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:06.735069 5528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:06.735321 5521 server_base.cc:1034] running on GCE node
I20260806 06:35:06.735533 5521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:35:06.735764 5521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:35:06.736891 5521 hybrid_clock.cc:648] HybridClock initialized: now 1785998106736877 us; error 28 us; skew 500 ppm
Aug 06 06:35:06 dist-test-slave-6zbq krb5kdc[5503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998106, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:35:06.739180 5521 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:35:06.739559 5521 webserver.cc:460] Webserver started at http://127.0.0.1:41561/ using document root <none> and password file <none>
I20260806 06:35:06.739742 5521 fs_manager.cc:359] Metadata directory not provided
I20260806 06:35:06.739789 5521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:35:06.739905 5521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:35:06.740828 5521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785997861828271-7341-0/minicluster-data/master-0/data/instance:
uuid: "a07dcf3712b14fbabf63cb8159ff7463"
format_stamp: "Formatted at 2026-08-06 06:35:06 on dist-test-slave-6zbq"
server_key: "4116fc71a19b6542be0c952ad0a45df7"
server_key_iv: "7304b673a37f73ac91b17c65c3f231af"
server_key_version: "encryptionkey@0"
I20260806 06:35:06.741163 5521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785997861828271-7341-0/minicluster-data/master-0/wal/instance:
uuid: "a07dcf3712b14fbabf63cb8159ff7463"
format_stamp: "Formatted at 2026-08-06 06:35:06 on dist-test-slave-6zbq"
server_key: "4116fc71a19b6542be0c952ad0a45df7"
server_key_iv: "7304b673a37f73ac91b17c65c3f231af"
server_key_version: "encryptionkey@0"
I20260806 06:35:06.742275 5521 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:35:06.742921 5542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:06.743063 5521 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:06.743150 5521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "a07dcf3712b14fbabf63cb8159ff7463"
format_stamp: "Formatted at 2026-08-06 06:35:06 on dist-test-slave-6zbq"
server_key: "4116fc71a19b6542be0c952ad0a45df7"
server_key_iv: "7304b673a37f73ac91b17c65c3f231af"
server_key_version: "encryptionkey@0"
I20260806 06:35:06.743206 5521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:35:06.746244 5521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:35:06.748188 5521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:35:06.748307 5521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:35:06.751927 5521 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46527
I20260806 06:35:06.751973 5611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46527 every 8 connection(s)
I20260806 06:35:06.752242 5521 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:35:06.753101 5612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:35:06.755108 5612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a07dcf3712b14fbabf63cb8159ff7463: Bootstrap starting.
I20260806 06:35:06.755590 5612 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a07dcf3712b14fbabf63cb8159ff7463: Neither blocks nor log segments found. Creating new log.
I20260806 06:35:06.755812 5612 log.cc:824] T 00000000000000000000000000000000 P a07dcf3712b14fbabf63cb8159ff7463: Log is configured to *not* fsync() on all Append() calls
I20260806 06:35:06.756392 5612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a07dcf3712b14fbabf63cb8159ff7463: No bootstrap required, opened a new log
I20260806 06:35:06.757839 5612 raft_consensus.cc:348] T 00000000000000000000000000000000 P a07dcf3712b14fbabf63cb8159ff7463 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a07dcf3712b14fbabf63cb8159ff7463" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46527 } }
I20260806 06:35:06.757958 5612 raft_consensus.cc:374] T 00000000000000000000000000000000 P a07dcf3712b14fbabf63cb8159ff7463 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:35:06.757978 5612 raft_consensus.cc:729] T 00000000000000000000000000000000 P a07dcf3712b14fbabf63cb8159ff7463 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a07dcf3712b14fbabf63cb8159ff7463, State: Initialized, Role: FOLLOWER
I20260806 06:35:06.758080 5612 consensus_queue.cc:260] T 00000000000000000000000000000000 P a07dcf3712b14fbabf63cb8159ff7463 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a07dcf3712b14fbabf63cb8159ff7463" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46527 } }
I20260806 06:35:06.758149 5612 raft_consensus.cc:388] T 00000000000000000000000000000000 P a07dcf3712b14fbabf63cb8159ff7463 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:35:06.758175 5612 raft_consensus.cc:482] T 00000000000000000000000000000000 P a07dcf3712b14fbabf63cb8159ff7463 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:35:06.758206 5612 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a07dcf3712b14fbabf63cb8159ff7463 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:35:06.758713 5612 raft_consensus.cc:504] T 00000000000000000000000000000000 P a07dcf3712b14fbabf63cb8159ff7463 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a07dcf3712b14fbabf63cb8159ff7463" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46527 } }
I20260806 06:35:06.758798 5612 leader_election.cc:302] T 00000000000000000000000000000000 P a07dcf3712b14fbabf63cb8159ff7463 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a07dcf3712b14fbabf63cb8159ff7463; no voters:
I20260806 06:35:06.758917 5612 leader_election.cc:288] T 00000000000000000000000000000000 P a07dcf3712b14fbabf63cb8159ff7463 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:35:06.758970 5617 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a07dcf3712b14fbabf63cb8159ff7463 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:35:06.759119 5612 sys_catalog.cc:564] T 00000000000000000000000000000000 P a07dcf3712b14fbabf63cb8159ff7463 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:35:06.759137 5617 raft_consensus.cc:686] T 00000000000000000000000000000000 P a07dcf3712b14fbabf63cb8159ff7463 [term 1 LEADER]: Becoming Leader. State: Replica: a07dcf3712b14fbabf63cb8159ff7463, State: Running, Role: LEADER
I20260806 06:35:06.759229 5617 consensus_queue.cc:237] T 00000000000000000000000000000000 P a07dcf3712b14fbabf63cb8159ff7463 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a07dcf3712b14fbabf63cb8159ff7463" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46527 } }
I20260806 06:35:06.759778 5619 sys_catalog.cc:455] T 00000000000000000000000000000000 P a07dcf3712b14fbabf63cb8159ff7463 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a07dcf3712b14fbabf63cb8159ff7463. Latest consensus state: current_term: 1 leader_uuid: "a07dcf3712b14fbabf63cb8159ff7463" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a07dcf3712b14fbabf63cb8159ff7463" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46527 } } }
I20260806 06:35:06.759786 5618 sys_catalog.cc:455] T 00000000000000000000000000000000 P a07dcf3712b14fbabf63cb8159ff7463 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a07dcf3712b14fbabf63cb8159ff7463" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a07dcf3712b14fbabf63cb8159ff7463" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46527 } } }
I20260806 06:35:06.759865 5619 sys_catalog.cc:458] T 00000000000000000000000000000000 P a07dcf3712b14fbabf63cb8159ff7463 [sys.catalog]: This master's current role is: LEADER
I20260806 06:35:06.759881 5618 sys_catalog.cc:458] T 00000000000000000000000000000000 P a07dcf3712b14fbabf63cb8159ff7463 [sys.catalog]: This master's current role is: LEADER
I20260806 06:35:06.760043 5629 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:35:06.760717 5629 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:35:06.761313 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 5521
I20260806 06:35:06.761428 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785997861828271-7341-0/minicluster-data/master-0/wal/instance
I20260806 06:35:06.761566 7341 external_mini_cluster.cc:1428] Setting key 6b3cd65b8bb14f689426bf00fa8e77dd
I20260806 06:35:06.762507 5629 catalog_manager.cc:1347] Generated new cluster ID: 10d876bafdcc45f3ba3dea476c53d081
I20260806 06:35:06.762552 5629 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:35:06.767526 5629 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:35:06.767666 5629 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:35:06.768158 5640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:06.761929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49513 (local address 127.0.0.1:46527)
0806 06:35:06.766635 (+ 4706us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:06.766645 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:06.766681 (+ 36us) server_negotiation.cc:407] Connection header received
0806 06:35:06.766812 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:06.766819 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:06.766850 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:06.766975 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:06.767729 (+ 754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:06.767733 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:06.767733 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:35:06.767737 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:06.767788 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:35:06.767803 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:06.767871 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:06.767943 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 06:35:06.767979 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4646,"thread_start_us":58,"threads_started":1}
I20260806 06:35:06.768944 7341 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:35:06.771211 5640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:06.770409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49705 (local address 127.0.0.1:46527)
0806 06:35:06.770440 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:06.770442 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:06.770450 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:35:06.770491 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:06.770492 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:06.770496 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:06.770557 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:06.771035 (+ 478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:06.771035 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:06.771036 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:35:06.771039 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:06.771049 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:35:06.771053 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:06.771100 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:06.771142 (+ 42us) server_negotiation.cc:299] Negotiation successful
0806 06:35:06.771153 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 06:35:06.774982 5629 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a07dcf3712b14fbabf63cb8159ff7463: Generated new TSK 0
I20260806 06:35:06.775131 5629 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:35:06.779811 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 5521
2026-08-06T06:35:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1373 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785997861828271-7341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:35:06 dist-test-slave-6zbq krb5kdc[5655](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:35:06 dist-test-slave-6zbq krb5kdc[5655](info): set up 2 sockets
Aug 06 06:35:06 dist-test-slave-6zbq krb5kdc[5655](info): commencing operation
krb5kdc: starting...
W20260806 06:35:07.891399 7341 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Aug 06 06:35:07 dist-test-slave-6zbq krb5kdc[5655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998107, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:35:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:35:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785997861828271-7341-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785997861828271-7341-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:35:07.995163 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40777
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785997861828271-7341-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40777
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:35:08.072636 5676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:35:08.072813 5676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:35:08.072844 5676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:35:08.073918 5676 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:35:08.073958 5676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:35:08.073977 5676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:35:08.073993 5676 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:35:08.074018 5676 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:35:08.075469 5676 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785997861828271-7341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40777
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785997861828271-7341-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5676
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:35:08.075690 5676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:35:08.075883 5676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:35:08.080205 5690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:08.080229 5687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:08.080230 5686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:08.080351 5676 server_base.cc:1034] running on GCE node
I20260806 06:35:08.080621 5676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:35:08.080870 5676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:35:08.082020 5676 hybrid_clock.cc:648] HybridClock initialized: now 1785998108082005 us; error 27 us; skew 500 ppm
Aug 06 06:35:08 dist-test-slave-6zbq krb5kdc[5655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998108, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:35:08.084273 5676 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:35:08.084687 5676 webserver.cc:460] Webserver started at http://127.0.0.1:34613/ using document root <none> and password file <none>
I20260806 06:35:08.084903 5676 fs_manager.cc:359] Metadata directory not provided
I20260806 06:35:08.084949 5676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:35:08.085068 5676 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:35:08.085992 5676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785997861828271-7341-0/minicluster-data/master-0/data/instance:
uuid: "74003c4b110949beb053e6d93eef29b6"
format_stamp: "Formatted at 2026-08-06 06:35:08 on dist-test-slave-6zbq"
server_key: "85f4c6a07e06b2628d5d1b95fb51475c"
server_key_iv: "8c39dedf2c93a7b128ecb831e5bfe593"
server_key_version: "encryptionkey@0"
I20260806 06:35:08.086292 5676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785997861828271-7341-0/minicluster-data/master-0/wal/instance:
uuid: "74003c4b110949beb053e6d93eef29b6"
format_stamp: "Formatted at 2026-08-06 06:35:08 on dist-test-slave-6zbq"
server_key: "85f4c6a07e06b2628d5d1b95fb51475c"
server_key_iv: "8c39dedf2c93a7b128ecb831e5bfe593"
server_key_version: "encryptionkey@0"
I20260806 06:35:08.087471 5676 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:35:08.088089 5698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:08.088239 5676 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:08.088320 5676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "74003c4b110949beb053e6d93eef29b6"
format_stamp: "Formatted at 2026-08-06 06:35:08 on dist-test-slave-6zbq"
server_key: "85f4c6a07e06b2628d5d1b95fb51475c"
server_key_iv: "8c39dedf2c93a7b128ecb831e5bfe593"
server_key_version: "encryptionkey@0"
I20260806 06:35:08.088379 5676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:35:08.098668 5676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:35:08.100162 5676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:35:08.100279 5676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:35:08.103843 5676 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40777
I20260806 06:35:08.103886 5766 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40777 every 8 connection(s)
I20260806 06:35:08.104154 5676 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:35:08.105033 5767 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:35:08.107084 5767 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74003c4b110949beb053e6d93eef29b6: Bootstrap starting.
I20260806 06:35:08.107607 5767 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 74003c4b110949beb053e6d93eef29b6: Neither blocks nor log segments found. Creating new log.
I20260806 06:35:08.107851 5767 log.cc:824] T 00000000000000000000000000000000 P 74003c4b110949beb053e6d93eef29b6: Log is configured to *not* fsync() on all Append() calls
I20260806 06:35:08.108448 5767 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74003c4b110949beb053e6d93eef29b6: No bootstrap required, opened a new log
I20260806 06:35:08.108983 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 5676
I20260806 06:35:08.109107 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785997861828271-7341-0/minicluster-data/master-0/wal/instance
I20260806 06:35:08.109287 7341 external_mini_cluster.cc:1428] Setting key afdeec8a542c9848a77731bfd17b6d76
I20260806 06:35:08.110484 5767 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74003c4b110949beb053e6d93eef29b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74003c4b110949beb053e6d93eef29b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40777 } }
I20260806 06:35:08.110616 5767 raft_consensus.cc:374] T 00000000000000000000000000000000 P 74003c4b110949beb053e6d93eef29b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:35:08.110648 5767 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74003c4b110949beb053e6d93eef29b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74003c4b110949beb053e6d93eef29b6, State: Initialized, Role: FOLLOWER
I20260806 06:35:08.110767 5767 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74003c4b110949beb053e6d93eef29b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74003c4b110949beb053e6d93eef29b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40777 } }
I20260806 06:35:08.110853 5767 raft_consensus.cc:388] T 00000000000000000000000000000000 P 74003c4b110949beb053e6d93eef29b6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:35:08.110884 5767 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74003c4b110949beb053e6d93eef29b6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:35:08.110934 5767 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74003c4b110949beb053e6d93eef29b6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:35:08.111559 5767 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74003c4b110949beb053e6d93eef29b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74003c4b110949beb053e6d93eef29b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40777 } }
I20260806 06:35:08.111665 5767 leader_election.cc:302] T 00000000000000000000000000000000 P 74003c4b110949beb053e6d93eef29b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 74003c4b110949beb053e6d93eef29b6; no voters:
I20260806 06:35:08.111825 5767 leader_election.cc:288] T 00000000000000000000000000000000 P 74003c4b110949beb053e6d93eef29b6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:35:08.111863 5774 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74003c4b110949beb053e6d93eef29b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:35:08.112004 5774 raft_consensus.cc:686] T 00000000000000000000000000000000 P 74003c4b110949beb053e6d93eef29b6 [term 1 LEADER]: Becoming Leader. State: Replica: 74003c4b110949beb053e6d93eef29b6, State: Running, Role: LEADER
I20260806 06:35:08.112067 5767 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74003c4b110949beb053e6d93eef29b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:35:08.112175 5774 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74003c4b110949beb053e6d93eef29b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74003c4b110949beb053e6d93eef29b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40777 } }
I20260806 06:35:08.113240 5777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74003c4b110949beb053e6d93eef29b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74003c4b110949beb053e6d93eef29b6. Latest consensus state: current_term: 1 leader_uuid: "74003c4b110949beb053e6d93eef29b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74003c4b110949beb053e6d93eef29b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40777 } } }
I20260806 06:35:08.113327 5777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74003c4b110949beb053e6d93eef29b6 [sys.catalog]: This master's current role is: LEADER
I20260806 06:35:08.113432 5778 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74003c4b110949beb053e6d93eef29b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "74003c4b110949beb053e6d93eef29b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74003c4b110949beb053e6d93eef29b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40777 } } }
I20260806 06:35:08.113953 5778 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74003c4b110949beb053e6d93eef29b6 [sys.catalog]: This master's current role is: LEADER
W20260806 06:35:08.114068 5801 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 74003c4b110949beb053e6d93eef29b6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:35:08.114110 5801 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:35:08.114238 5805 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:35:08.114440 5805 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 06:35:08 dist-test-slave-6zbq krb5kdc[5655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998107, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 06:35:08.118726 5805 catalog_manager.cc:1347] Generated new cluster ID: a3a4ac7b29184e359adc6afee8e5adf5
I20260806 06:35:08.118777 5805 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:35:08.120082 5772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:08.109725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:49827 (local address 127.0.0.1:40777)
0806 06:35:08.109887 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:08.109896 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:08.109920 (+ 24us) server_negotiation.cc:407] Connection header received
0806 06:35:08.110625 (+ 705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:08.110632 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:08.110692 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:08.110812 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:08.111556 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:08.112818 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:08.113808 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:08.114390 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:08.114503 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:08.116721 (+ 2218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:08.116725 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:08.116725 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:08.116736 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:08.118448 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:08.118907 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:08.118910 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:08.118910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:08.119008 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:08.119267 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:08.119268 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:08.119268 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:08.119529 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:08.119655 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:08.119804 (+ 149us) server_negotiation.cc:299] Negotiation successful
0806 06:35:08.119901 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":60,"threads_started":1}
I20260806 06:35:08.121030 7341 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:35:08.133070 5772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:08.122368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:51899 (local address 127.0.0.1:40777)
0806 06:35:08.122398 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:08.122400 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:08.126533 (+ 4133us) server_negotiation.cc:407] Connection header received
0806 06:35:08.126580 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:08.126581 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:08.126590 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:08.126643 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:08.127209 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:08.127946 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:08.128914 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:08.129205 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:08.129259 (+ 54us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:08.130579 (+ 1320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:08.130580 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:08.130581 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:08.130587 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:08.131998 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:08.132383 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:08.132384 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:08.132384 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:08.132432 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:08.132623 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:08.132624 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:08.132624 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:08.132726 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:08.132803 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:08.132935 (+ 132us) server_negotiation.cc:299] Negotiation successful
0806 06:35:08.132989 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 06:35:08.170595 5805 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:35:08.171300 5805 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:35:08.183648 5805 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 74003c4b110949beb053e6d93eef29b6: Generated new TSK 0
I20260806 06:35:08.183816 5805 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:35:08.191875 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 5676
W20260806 06:35:08.205365 5651 connection.cc:535] client connection to 127.0.0.1:40777 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40777): Connection reset by peer (error 104)
2026-08-06T06:35:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1404 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785997861828271-7341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:35:08 dist-test-slave-6zbq krb5kdc[5818](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:35:08 dist-test-slave-6zbq krb5kdc[5818](info): set up 2 sockets
Aug 06 06:35:08 dist-test-slave-6zbq krb5kdc[5818](info): commencing operation
krb5kdc: starting...
W20260806 06:35:09.308706 7341 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Aug 06 06:35:09 dist-test-slave-6zbq krb5kdc[5818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998109, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:35:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:35:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785997861828271-7341-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785997861828271-7341-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:35:09.418151 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46441
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785997861828271-7341-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46441
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:35:09.496116 5840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:35:09.496316 5840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:35:09.496347 5840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:35:09.497444 5840 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:35:09.497484 5840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:35:09.497502 5840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:35:09.497519 5840 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:35:09.497534 5840 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:35:09.498982 5840 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785997861828271-7341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46441
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785997861828271-7341-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46441
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:35:09.499207 5840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:35:09.499398 5840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:35:09.503892 5847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:09.503988 5840 server_base.cc:1034] running on GCE node
W20260806 06:35:09.503903 5851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:09.503907 5849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:09.504294 5840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:35:09.504557 5840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:35:09.505692 5840 hybrid_clock.cc:648] HybridClock initialized: now 1785998109505679 us; error 28 us; skew 500 ppm
Aug 06 06:35:09 dist-test-slave-6zbq krb5kdc[5818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998109, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:35:09.507901 5840 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:35:09.508296 5840 webserver.cc:460] Webserver started at http://127.0.0.1:33753/ using document root <none> and password file <none>
I20260806 06:35:09.508482 5840 fs_manager.cc:359] Metadata directory not provided
I20260806 06:35:09.508527 5840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:35:09.508648 5840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:35:09.509929 5840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785997861828271-7341-0/minicluster-data/master-0/data/instance:
uuid: "19d95b8f33954b50ba8698c95c9d64ca"
format_stamp: "Formatted at 2026-08-06 06:35:09 on dist-test-slave-6zbq"
server_key: "025227c2337eb2eaeb16f973c9618e62"
server_key_iv: "c3585003814343c0164a15e89d0b0364"
server_key_version: "encryptionkey@0"
I20260806 06:35:09.510368 5840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785997861828271-7341-0/minicluster-data/master-0/wal/instance:
uuid: "19d95b8f33954b50ba8698c95c9d64ca"
format_stamp: "Formatted at 2026-08-06 06:35:09 on dist-test-slave-6zbq"
server_key: "025227c2337eb2eaeb16f973c9618e62"
server_key_iv: "c3585003814343c0164a15e89d0b0364"
server_key_version: "encryptionkey@0"
I20260806 06:35:09.511579 5840 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:35:09.512233 5858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:09.512394 5840 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:35:09.512475 5840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "19d95b8f33954b50ba8698c95c9d64ca"
format_stamp: "Formatted at 2026-08-06 06:35:09 on dist-test-slave-6zbq"
server_key: "025227c2337eb2eaeb16f973c9618e62"
server_key_iv: "c3585003814343c0164a15e89d0b0364"
server_key_version: "encryptionkey@0"
I20260806 06:35:09.512527 5840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:35:09.526042 5840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:35:09.527679 5840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:35:09.527778 5840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:35:09.531376 5840 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46441
I20260806 06:35:09.531419 5929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46441 every 8 connection(s)
I20260806 06:35:09.531711 5840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:35:09.532102 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 5840
I20260806 06:35:09.532217 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785997861828271-7341-0/minicluster-data/master-0/wal/instance
I20260806 06:35:09.532388 7341 external_mini_cluster.cc:1428] Setting key 28780de8195498c0c13cd359e34ba448
I20260806 06:35:09.532685 5931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:35:09.535545 5931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19d95b8f33954b50ba8698c95c9d64ca: Bootstrap starting.
I20260806 06:35:09.536201 5931 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 19d95b8f33954b50ba8698c95c9d64ca: Neither blocks nor log segments found. Creating new log.
I20260806 06:35:09.536468 5931 log.cc:824] T 00000000000000000000000000000000 P 19d95b8f33954b50ba8698c95c9d64ca: Log is configured to *not* fsync() on all Append() calls
I20260806 06:35:09.537092 5931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19d95b8f33954b50ba8698c95c9d64ca: No bootstrap required, opened a new log
I20260806 06:35:09.537753 5934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:09.532843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:53717 (local address 127.0.0.1:46441)
0806 06:35:09.533003 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:09.533010 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:09.533034 (+ 24us) server_negotiation.cc:407] Connection header received
0806 06:35:09.533681 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:09.533688 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:09.533735 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:09.533838 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:09.534108 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:09.535009 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:09.536068 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:09.536477 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:09.536573 (+ 96us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:09.537214 (+ 641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:09.537218 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:09.537218 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:35:09.537358 (+ 140us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:09.537387 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:35:09.537396 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:09.537462 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:09.537554 (+ 92us) server_negotiation.cc:299] Negotiation successful
0806 06:35:09.537579 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":66,"threads_started":1}
I20260806 06:35:09.538779 7341 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:35:09.539081 5931 raft_consensus.cc:348] T 00000000000000000000000000000000 P 19d95b8f33954b50ba8698c95c9d64ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19d95b8f33954b50ba8698c95c9d64ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46441 } }
I20260806 06:35:09.539206 5931 raft_consensus.cc:374] T 00000000000000000000000000000000 P 19d95b8f33954b50ba8698c95c9d64ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:35:09.539239 5931 raft_consensus.cc:729] T 00000000000000000000000000000000 P 19d95b8f33954b50ba8698c95c9d64ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19d95b8f33954b50ba8698c95c9d64ca, State: Initialized, Role: FOLLOWER
I20260806 06:35:09.539351 5931 consensus_queue.cc:260] T 00000000000000000000000000000000 P 19d95b8f33954b50ba8698c95c9d64ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19d95b8f33954b50ba8698c95c9d64ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46441 } }
I20260806 06:35:09.539408 5931 raft_consensus.cc:388] T 00000000000000000000000000000000 P 19d95b8f33954b50ba8698c95c9d64ca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:35:09.539436 5931 raft_consensus.cc:482] T 00000000000000000000000000000000 P 19d95b8f33954b50ba8698c95c9d64ca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:35:09.539481 5931 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 19d95b8f33954b50ba8698c95c9d64ca [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:35:09.539963 5931 raft_consensus.cc:504] T 00000000000000000000000000000000 P 19d95b8f33954b50ba8698c95c9d64ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19d95b8f33954b50ba8698c95c9d64ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46441 } }
I20260806 06:35:09.540055 5931 leader_election.cc:302] T 00000000000000000000000000000000 P 19d95b8f33954b50ba8698c95c9d64ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 19d95b8f33954b50ba8698c95c9d64ca; no voters:
I20260806 06:35:09.540221 5931 leader_election.cc:288] T 00000000000000000000000000000000 P 19d95b8f33954b50ba8698c95c9d64ca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:35:09.540257 5943 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 19d95b8f33954b50ba8698c95c9d64ca [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:35:09.540454 5943 raft_consensus.cc:686] T 00000000000000000000000000000000 P 19d95b8f33954b50ba8698c95c9d64ca [term 1 LEADER]: Becoming Leader. State: Replica: 19d95b8f33954b50ba8698c95c9d64ca, State: Running, Role: LEADER
I20260806 06:35:09.540545 5931 sys_catalog.cc:564] T 00000000000000000000000000000000 P 19d95b8f33954b50ba8698c95c9d64ca [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:35:09.540591 5943 consensus_queue.cc:237] T 00000000000000000000000000000000 P 19d95b8f33954b50ba8698c95c9d64ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19d95b8f33954b50ba8698c95c9d64ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46441 } }
I20260806 06:35:09.540930 5943 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19d95b8f33954b50ba8698c95c9d64ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 19d95b8f33954b50ba8698c95c9d64ca. Latest consensus state: current_term: 1 leader_uuid: "19d95b8f33954b50ba8698c95c9d64ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19d95b8f33954b50ba8698c95c9d64ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46441 } } }
I20260806 06:35:09.540985 5944 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19d95b8f33954b50ba8698c95c9d64ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "19d95b8f33954b50ba8698c95c9d64ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19d95b8f33954b50ba8698c95c9d64ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46441 } } }
I20260806 06:35:09.541021 5943 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19d95b8f33954b50ba8698c95c9d64ca [sys.catalog]: This master's current role is: LEADER
I20260806 06:35:09.541036 5944 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19d95b8f33954b50ba8698c95c9d64ca [sys.catalog]: This master's current role is: LEADER
I20260806 06:35:09.541749 5955 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:35:09.542212 5955 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:35:09.543339 5934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:09.540041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:56425 (local address 127.0.0.1:46441)
0806 06:35:09.540084 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:09.540086 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:09.540195 (+ 109us) server_negotiation.cc:407] Connection header received
0806 06:35:09.540281 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:09.540282 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:09.540291 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:09.540345 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:09.540506 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:09.541173 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:09.542137 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:09.542429 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:09.542486 (+ 57us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:09.543072 (+ 586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:09.543074 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:09.543074 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:35:09.543078 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:09.543092 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:35:09.543096 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:09.543149 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:09.543261 (+ 112us) server_negotiation.cc:299] Negotiation successful
0806 06:35:09.543271 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 06:35:09.543728 5955 catalog_manager.cc:1347] Generated new cluster ID: d7b7624a3aeb460799921647b56fcdde
I20260806 06:35:09.543780 5955 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:35:09.553180 5955 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:35:09.553665 5955 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:35:09.557391 5955 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 19d95b8f33954b50ba8698c95c9d64ca: Generated new TSK 0
I20260806 06:35:09.557524 5955 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:35:09.559132 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 5840
W20260806 06:35:09.574824 5816 connection.cc:535] client connection to 127.0.0.1:46441 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46441): Connection reset by peer (error 104)
2026-08-06T06:35:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1369 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785997861828271-7341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:35:09 dist-test-slave-6zbq krb5kdc[5976](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:35:09 dist-test-slave-6zbq krb5kdc[5976](info): set up 2 sockets
Aug 06 06:35:09 dist-test-slave-6zbq krb5kdc[5976](info): commencing operation
krb5kdc: starting...
W20260806 06:35:10.670928 7341 mini_kdc.cc:121] Time spent starting KDC: real 1.083s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Aug 06 06:35:10 dist-test-slave-6zbq krb5kdc[5976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998110, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:35:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:35:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785997861828271-7341-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785997861828271-7341-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:35:10.776571 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44757
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785997861828271-7341-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44757
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:35:10.855054 5992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:35:10.855232 5992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:35:10.855260 5992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:35:10.856359 5992 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:35:10.856398 5992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:35:10.856412 5992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:35:10.856436 5992 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:35:10.856458 5992 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:35:10.857883 5992 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785997861828271-7341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44757
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785997861828271-7341-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44757
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5992
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:35:10.858094 5992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:35:10.858290 5992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:35:10.862869 6000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:10.862859 6002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:10.862906 6004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:10.863051 5992 server_base.cc:1034] running on GCE node
I20260806 06:35:10.863261 5992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:35:10.863530 5992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:35:10.864666 5992 hybrid_clock.cc:648] HybridClock initialized: now 1785998110864652 us; error 28 us; skew 500 ppm
Aug 06 06:35:10 dist-test-slave-6zbq krb5kdc[5976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998110, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:35:10.870986 5992 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:35:10.871399 5992 webserver.cc:460] Webserver started at http://127.0.0.1:41371/ using document root <none> and password file <none>
I20260806 06:35:10.871600 5992 fs_manager.cc:359] Metadata directory not provided
I20260806 06:35:10.871645 5992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:35:10.871760 5992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:35:10.872684 5992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785997861828271-7341-0/minicluster-data/master-0/data/instance:
uuid: "8fd79ccd075b468db00816626d9b0bd9"
format_stamp: "Formatted at 2026-08-06 06:35:10 on dist-test-slave-6zbq"
server_key: "7901a68d9a306a57d89b56d87174927b"
server_key_iv: "441f3288dc3e8621cd02bbf554426b50"
server_key_version: "encryptionkey@0"
I20260806 06:35:10.872992 5992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785997861828271-7341-0/minicluster-data/master-0/wal/instance:
uuid: "8fd79ccd075b468db00816626d9b0bd9"
format_stamp: "Formatted at 2026-08-06 06:35:10 on dist-test-slave-6zbq"
server_key: "7901a68d9a306a57d89b56d87174927b"
server_key_iv: "441f3288dc3e8621cd02bbf554426b50"
server_key_version: "encryptionkey@0"
I20260806 06:35:10.874122 5992 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:35:10.874828 6013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:10.874971 5992 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:10.875051 5992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "8fd79ccd075b468db00816626d9b0bd9"
format_stamp: "Formatted at 2026-08-06 06:35:10 on dist-test-slave-6zbq"
server_key: "7901a68d9a306a57d89b56d87174927b"
server_key_iv: "441f3288dc3e8621cd02bbf554426b50"
server_key_version: "encryptionkey@0"
I20260806 06:35:10.875115 5992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:35:10.878109 5992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:35:10.880021 5992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:35:10.880144 5992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:35:10.883711 5992 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44757
I20260806 06:35:10.883749 6085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44757 every 8 connection(s)
I20260806 06:35:10.884006 5992 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:35:10.884855 6086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:35:10.886862 6086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fd79ccd075b468db00816626d9b0bd9: Bootstrap starting.
I20260806 06:35:10.887377 6086 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8fd79ccd075b468db00816626d9b0bd9: Neither blocks nor log segments found. Creating new log.
I20260806 06:35:10.887609 6086 log.cc:824] T 00000000000000000000000000000000 P 8fd79ccd075b468db00816626d9b0bd9: Log is configured to *not* fsync() on all Append() calls
I20260806 06:35:10.888195 6086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fd79ccd075b468db00816626d9b0bd9: No bootstrap required, opened a new log
I20260806 06:35:10.889644 6086 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8fd79ccd075b468db00816626d9b0bd9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fd79ccd075b468db00816626d9b0bd9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44757 } }
I20260806 06:35:10.889766 6086 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8fd79ccd075b468db00816626d9b0bd9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:35:10.889799 6086 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8fd79ccd075b468db00816626d9b0bd9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd79ccd075b468db00816626d9b0bd9, State: Initialized, Role: FOLLOWER
I20260806 06:35:10.889925 6086 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8fd79ccd075b468db00816626d9b0bd9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fd79ccd075b468db00816626d9b0bd9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44757 } }
I20260806 06:35:10.889983 6086 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8fd79ccd075b468db00816626d9b0bd9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:35:10.890004 6086 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8fd79ccd075b468db00816626d9b0bd9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:35:10.890048 6086 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8fd79ccd075b468db00816626d9b0bd9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:35:10.890461 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 5992
I20260806 06:35:10.890573 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785997861828271-7341-0/minicluster-data/master-0/wal/instance
I20260806 06:35:10.890575 6086 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8fd79ccd075b468db00816626d9b0bd9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fd79ccd075b468db00816626d9b0bd9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44757 } }
I20260806 06:35:10.890669 6086 leader_election.cc:302] T 00000000000000000000000000000000 P 8fd79ccd075b468db00816626d9b0bd9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8fd79ccd075b468db00816626d9b0bd9; no voters:
I20260806 06:35:10.890743 7341 external_mini_cluster.cc:1428] Setting key 532b8ca7b01a407df2b17cf25b5eb851
I20260806 06:35:10.890808 6086 leader_election.cc:288] T 00000000000000000000000000000000 P 8fd79ccd075b468db00816626d9b0bd9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:35:10.890847 6090 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8fd79ccd075b468db00816626d9b0bd9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:35:10.891007 6090 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8fd79ccd075b468db00816626d9b0bd9 [term 1 LEADER]: Becoming Leader. State: Replica: 8fd79ccd075b468db00816626d9b0bd9, State: Running, Role: LEADER
I20260806 06:35:10.891041 6086 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8fd79ccd075b468db00816626d9b0bd9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:35:10.891090 6090 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8fd79ccd075b468db00816626d9b0bd9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fd79ccd075b468db00816626d9b0bd9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44757 } }
I20260806 06:35:10.891407 6090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fd79ccd075b468db00816626d9b0bd9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8fd79ccd075b468db00816626d9b0bd9. Latest consensus state: current_term: 1 leader_uuid: "8fd79ccd075b468db00816626d9b0bd9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fd79ccd075b468db00816626d9b0bd9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44757 } } }
I20260806 06:35:10.891482 6090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fd79ccd075b468db00816626d9b0bd9 [sys.catalog]: This master's current role is: LEADER
I20260806 06:35:10.891623 6091 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fd79ccd075b468db00816626d9b0bd9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8fd79ccd075b468db00816626d9b0bd9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fd79ccd075b468db00816626d9b0bd9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44757 } } }
I20260806 06:35:10.891695 6101 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:35:10.891717 6091 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fd79ccd075b468db00816626d9b0bd9 [sys.catalog]: This master's current role is: LEADER
I20260806 06:35:10.892242 6101 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:35:10.893471 6095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:10.891250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:48971 (local address 127.0.0.1:44757)
0806 06:35:10.891422 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:10.891431 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:10.891581 (+ 150us) server_negotiation.cc:407] Connection header received
0806 06:35:10.891694 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:10.891832 (+ 138us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:10.891874 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:10.892631 (+ 757us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:10.892911 (+ 280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:10.892914 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:10.892914 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:35:10.892918 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:10.892954 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:35:10.892967 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:10.893235 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:10.893266 (+ 31us) server_negotiation.cc:299] Negotiation successful
0806 06:35:10.893308 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":79,"threads_started":1}
I20260806 06:35:10.894286 6101 catalog_manager.cc:1347] Generated new cluster ID: 610f69a19aaf4d47a7ed0035b3360851
I20260806 06:35:10.894356 6101 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:35:10.894544 7341 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:35:10.896644 6095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:10.895738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:46053 (local address 127.0.0.1:44757)
0806 06:35:10.895770 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:10.895771 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:10.895873 (+ 102us) server_negotiation.cc:407] Connection header received
0806 06:35:10.895937 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:10.895938 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:10.895943 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:10.895985 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:10.896449 (+ 464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:10.896450 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:10.896451 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:35:10.896454 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:10.896467 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:35:10.896471 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:10.896516 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:10.896564 (+ 48us) server_negotiation.cc:299] Negotiation successful
0806 06:35:10.896577 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:35:10.906687 6101 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:35:10.906819 6101 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:35:10.911463 6101 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8fd79ccd075b468db00816626d9b0bd9: Generated new TSK 0
I20260806 06:35:10.911602 6101 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:35:10.917430 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 5992
2026-08-06T06:35:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1354 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785997861828271-7341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:35:10 dist-test-slave-6zbq krb5kdc[6134](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:35:10 dist-test-slave-6zbq krb5kdc[6134](info): set up 2 sockets
Aug 06 06:35:10 dist-test-slave-6zbq krb5kdc[6134](info): commencing operation
krb5kdc: starting...
W20260806 06:35:12.023602 7341 mini_kdc.cc:121] Time spent starting KDC: real 1.081s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Aug 06 06:35:12 dist-test-slave-6zbq krb5kdc[6134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998112, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:35:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:35:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785997861828271-7341-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785997861828271-7341-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:35:12.128729 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35557
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785997861828271-7341-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35557
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:35:12.206791 6151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:35:12.206969 6151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:35:12.206990 6151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:35:12.208245 6151 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:35:12.208295 6151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:35:12.208308 6151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:35:12.208319 6151 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:35:12.208330 6151 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:35:12.209862 6151 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785997861828271-7341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35557
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785997861828271-7341-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35557
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6151
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:35:12.210037 6151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:35:12.210209 6151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:35:12.214581 6161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:12.214637 6159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:12.214737 6151 server_base.cc:1034] running on GCE node
W20260806 06:35:12.214638 6164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:12.215013 6151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:35:12.215281 6151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:35:12.216420 6151 hybrid_clock.cc:648] HybridClock initialized: now 1785998112216405 us; error 26 us; skew 500 ppm
Aug 06 06:35:12 dist-test-slave-6zbq krb5kdc[6134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998112, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:35:12.218725 6151 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:35:12.220486 6151 webserver.cc:460] Webserver started at http://127.0.0.1:37779/ using document root <none> and password file <none>
I20260806 06:35:12.220738 6151 fs_manager.cc:359] Metadata directory not provided
I20260806 06:35:12.220788 6151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:35:12.220876 6151 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:35:12.221827 6151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785997861828271-7341-0/minicluster-data/master-0/data/instance:
uuid: "dcac86d3bdd44148a1c4c9b3d79c524e"
format_stamp: "Formatted at 2026-08-06 06:35:12 on dist-test-slave-6zbq"
server_key: "e2e70d2d1ac31577f81f2aab7065aaa8"
server_key_iv: "65570fc14e72e63161265e8e45f15384"
server_key_version: "encryptionkey@0"
I20260806 06:35:12.222110 6151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785997861828271-7341-0/minicluster-data/master-0/wal/instance:
uuid: "dcac86d3bdd44148a1c4c9b3d79c524e"
format_stamp: "Formatted at 2026-08-06 06:35:12 on dist-test-slave-6zbq"
server_key: "e2e70d2d1ac31577f81f2aab7065aaa8"
server_key_iv: "65570fc14e72e63161265e8e45f15384"
server_key_version: "encryptionkey@0"
I20260806 06:35:12.223335 6151 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:35:12.223958 6172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:12.224113 6151 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:35:12.224197 6151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "dcac86d3bdd44148a1c4c9b3d79c524e"
format_stamp: "Formatted at 2026-08-06 06:35:12 on dist-test-slave-6zbq"
server_key: "e2e70d2d1ac31577f81f2aab7065aaa8"
server_key_iv: "65570fc14e72e63161265e8e45f15384"
server_key_version: "encryptionkey@0"
I20260806 06:35:12.224259 6151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:35:12.239039 6151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:35:12.240693 6151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:35:12.240821 6151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:35:12.244453 6151 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35557
I20260806 06:35:12.244484 6249 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35557 every 8 connection(s)
I20260806 06:35:12.244773 6151 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:35:12.245661 6251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:35:12.247903 6251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcac86d3bdd44148a1c4c9b3d79c524e: Bootstrap starting.
I20260806 06:35:12.248457 6251 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dcac86d3bdd44148a1c4c9b3d79c524e: Neither blocks nor log segments found. Creating new log.
I20260806 06:35:12.248701 6251 log.cc:824] T 00000000000000000000000000000000 P dcac86d3bdd44148a1c4c9b3d79c524e: Log is configured to *not* fsync() on all Append() calls
I20260806 06:35:12.249415 6251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcac86d3bdd44148a1c4c9b3d79c524e: No bootstrap required, opened a new log
I20260806 06:35:12.251118 6251 raft_consensus.cc:348] T 00000000000000000000000000000000 P dcac86d3bdd44148a1c4c9b3d79c524e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcac86d3bdd44148a1c4c9b3d79c524e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35557 } }
I20260806 06:35:12.251273 6251 raft_consensus.cc:374] T 00000000000000000000000000000000 P dcac86d3bdd44148a1c4c9b3d79c524e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:35:12.251295 6251 raft_consensus.cc:729] T 00000000000000000000000000000000 P dcac86d3bdd44148a1c4c9b3d79c524e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcac86d3bdd44148a1c4c9b3d79c524e, State: Initialized, Role: FOLLOWER
I20260806 06:35:12.251411 6251 consensus_queue.cc:260] T 00000000000000000000000000000000 P dcac86d3bdd44148a1c4c9b3d79c524e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcac86d3bdd44148a1c4c9b3d79c524e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35557 } }
I20260806 06:35:12.251492 6251 raft_consensus.cc:388] T 00000000000000000000000000000000 P dcac86d3bdd44148a1c4c9b3d79c524e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:35:12.251519 6251 raft_consensus.cc:482] T 00000000000000000000000000000000 P dcac86d3bdd44148a1c4c9b3d79c524e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:35:12.251546 6251 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dcac86d3bdd44148a1c4c9b3d79c524e [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:35:12.252087 6251 raft_consensus.cc:504] T 00000000000000000000000000000000 P dcac86d3bdd44148a1c4c9b3d79c524e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcac86d3bdd44148a1c4c9b3d79c524e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35557 } }
I20260806 06:35:12.252184 6251 leader_election.cc:302] T 00000000000000000000000000000000 P dcac86d3bdd44148a1c4c9b3d79c524e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dcac86d3bdd44148a1c4c9b3d79c524e; no voters:
I20260806 06:35:12.252331 6251 leader_election.cc:288] T 00000000000000000000000000000000 P dcac86d3bdd44148a1c4c9b3d79c524e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:35:12.252400 6255 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dcac86d3bdd44148a1c4c9b3d79c524e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:35:12.252549 6255 raft_consensus.cc:686] T 00000000000000000000000000000000 P dcac86d3bdd44148a1c4c9b3d79c524e [term 1 LEADER]: Becoming Leader. State: Replica: dcac86d3bdd44148a1c4c9b3d79c524e, State: Running, Role: LEADER
I20260806 06:35:12.252624 6251 sys_catalog.cc:564] T 00000000000000000000000000000000 P dcac86d3bdd44148a1c4c9b3d79c524e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:35:12.252724 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 6151
I20260806 06:35:12.252718 6255 consensus_queue.cc:237] T 00000000000000000000000000000000 P dcac86d3bdd44148a1c4c9b3d79c524e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcac86d3bdd44148a1c4c9b3d79c524e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35557 } }
I20260806 06:35:12.252863 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785997861828271-7341-0/minicluster-data/master-0/wal/instance
I20260806 06:35:12.253037 7341 external_mini_cluster.cc:1428] Setting key c8cd270730e93f5dd23500815a4f8082
I20260806 06:35:12.253276 6256 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcac86d3bdd44148a1c4c9b3d79c524e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dcac86d3bdd44148a1c4c9b3d79c524e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcac86d3bdd44148a1c4c9b3d79c524e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35557 } } }
I20260806 06:35:12.253365 6256 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcac86d3bdd44148a1c4c9b3d79c524e [sys.catalog]: This master's current role is: LEADER
I20260806 06:35:12.253700 6258 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcac86d3bdd44148a1c4c9b3d79c524e [sys.catalog]: SysCatalogTable state changed. Reason: New leader dcac86d3bdd44148a1c4c9b3d79c524e. Latest consensus state: current_term: 1 leader_uuid: "dcac86d3bdd44148a1c4c9b3d79c524e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcac86d3bdd44148a1c4c9b3d79c524e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35557 } } }
I20260806 06:35:12.253726 6262 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:35:12.253806 6258 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcac86d3bdd44148a1c4c9b3d79c524e [sys.catalog]: This master's current role is: LEADER
I20260806 06:35:12.254428 6262 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:35:12.256227 6262 catalog_manager.cc:1347] Generated new cluster ID: fb313d2925dd4143a72b4d322c44f7b2
I20260806 06:35:12.256279 6262 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:35:12 dist-test-slave-6zbq krb5kdc[6134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998112, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 06:35:12.264016 6266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:12.253743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:59309 (local address 127.0.0.1:35557)
0806 06:35:12.253993 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:12.254005 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:12.254042 (+ 37us) server_negotiation.cc:407] Connection header received
0806 06:35:12.254172 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:12.254180 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:12.254243 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:12.255005 (+ 762us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:12.255231 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:12.256385 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:12.257109 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:12.257527 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:12.257628 (+ 101us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:12.260460 (+ 2832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:12.260464 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:12.260464 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:12.260475 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:12.262548 (+ 2073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:12.262928 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:12.262930 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:12.262931 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:12.263028 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:12.263232 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:12.263233 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:12.263234 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:12.263455 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:12.263605 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:12.263723 (+ 118us) server_negotiation.cc:299] Negotiation successful
0806 06:35:12.263821 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":75,"threads_started":1}
I20260806 06:35:12.264943 7341 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:35:12.273985 6262 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:35:12.274798 6262 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:35:12.276165 6266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:12.266367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:36095 (local address 127.0.0.1:35557)
0806 06:35:12.266404 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:12.266406 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:12.270579 (+ 4173us) server_negotiation.cc:407] Connection header received
0806 06:35:12.270649 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:12.270649 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:12.270660 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:12.270709 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:12.271208 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:12.271708 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:12.272360 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:12.272561 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:12.272616 (+ 55us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:12.273720 (+ 1104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:12.273722 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:12.273722 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:12.273729 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:12.275136 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:12.275474 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:12.275475 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:12.275476 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:12.275529 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:12.275715 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:12.275716 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:12.275716 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:12.275817 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:12.275918 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:12.276028 (+ 110us) server_negotiation.cc:299] Negotiation successful
0806 06:35:12.276083 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 06:35:12.279579 6262 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dcac86d3bdd44148a1c4c9b3d79c524e: Generated new TSK 0
I20260806 06:35:12.279748 6262 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:35:12.283603 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 6151
W20260806 06:35:12.297851 6127 connection.cc:535] client connection to 127.0.0.1:35557 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35557): Connection reset by peer (error 104)
2026-08-06T06:35:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1368 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785997861828271-7341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:35:12 dist-test-slave-6zbq krb5kdc[6289](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:35:12 dist-test-slave-6zbq krb5kdc[6289](info): set up 2 sockets
Aug 06 06:35:12 dist-test-slave-6zbq krb5kdc[6289](info): commencing operation
krb5kdc: starting...
W20260806 06:35:13.395604 7341 mini_kdc.cc:121] Time spent starting KDC: real 1.084s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Aug 06 06:35:13 dist-test-slave-6zbq krb5kdc[6289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998113, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:35:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:35:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785997861828271-7341-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785997861828271-7341-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:35:13.498873 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41743
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785997861828271-7341-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41743
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:35:13.576905 6306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:35:13.577093 6306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:35:13.577147 6306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:35:13.578225 6306 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:35:13.578263 6306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:35:13.578282 6306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:35:13.578298 6306 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:35:13.578313 6306 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:35:13.579752 6306 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785997861828271-7341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41743
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785997861828271-7341-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41743
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6306
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:35:13.579981 6306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:35:13.580178 6306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:35:13.584697 6317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:13.584836 6306 server_base.cc:1034] running on GCE node
W20260806 06:35:13.584741 6314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:13.584708 6312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:13.585170 6306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:35:13.585426 6306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:35:13.586556 6306 hybrid_clock.cc:648] HybridClock initialized: now 1785998113586537 us; error 33 us; skew 500 ppm
Aug 06 06:35:13 dist-test-slave-6zbq krb5kdc[6289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998113, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:35:13.588785 6306 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:35:13.589190 6306 webserver.cc:460] Webserver started at http://127.0.0.1:34321/ using document root <none> and password file <none>
I20260806 06:35:13.589368 6306 fs_manager.cc:359] Metadata directory not provided
I20260806 06:35:13.589409 6306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:35:13.589507 6306 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:35:13.590476 6306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785997861828271-7341-0/minicluster-data/master-0/data/instance:
uuid: "18f40e9ddffd474cbef80cbbc605c02e"
format_stamp: "Formatted at 2026-08-06 06:35:13 on dist-test-slave-6zbq"
server_key: "7755d8d7a82290e2d8681ece73fc7fb3"
server_key_iv: "7f1bb80ee256689d7abb16c37ce1f502"
server_key_version: "encryptionkey@0"
I20260806 06:35:13.590775 6306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785997861828271-7341-0/minicluster-data/master-0/wal/instance:
uuid: "18f40e9ddffd474cbef80cbbc605c02e"
format_stamp: "Formatted at 2026-08-06 06:35:13 on dist-test-slave-6zbq"
server_key: "7755d8d7a82290e2d8681ece73fc7fb3"
server_key_iv: "7f1bb80ee256689d7abb16c37ce1f502"
server_key_version: "encryptionkey@0"
I20260806 06:35:13.591917 6306 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:35:13.592519 6325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:13.592655 6306 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:35:13.592728 6306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "18f40e9ddffd474cbef80cbbc605c02e"
format_stamp: "Formatted at 2026-08-06 06:35:13 on dist-test-slave-6zbq"
server_key: "7755d8d7a82290e2d8681ece73fc7fb3"
server_key_iv: "7f1bb80ee256689d7abb16c37ce1f502"
server_key_version: "encryptionkey@0"
I20260806 06:35:13.592789 6306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:35:13.606483 6306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:35:13.608320 6306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:35:13.608436 6306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:35:13.612023 6306 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41743
I20260806 06:35:13.612057 6408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41743 every 8 connection(s)
I20260806 06:35:13.612332 6306 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:35:13.612579 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 6306
I20260806 06:35:13.612699 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785997861828271-7341-0/minicluster-data/master-0/wal/instance
I20260806 06:35:13.612876 7341 external_mini_cluster.cc:1428] Setting key 5d7ff2fd8208bac8f24234e459d65599
I20260806 06:35:13.613150 6409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:35:13.615635 6409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18f40e9ddffd474cbef80cbbc605c02e: Bootstrap starting.
I20260806 06:35:13.616158 6409 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 18f40e9ddffd474cbef80cbbc605c02e: Neither blocks nor log segments found. Creating new log.
I20260806 06:35:13.616406 6409 log.cc:824] T 00000000000000000000000000000000 P 18f40e9ddffd474cbef80cbbc605c02e: Log is configured to *not* fsync() on all Append() calls
I20260806 06:35:13.616971 6409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18f40e9ddffd474cbef80cbbc605c02e: No bootstrap required, opened a new log
I20260806 06:35:13.617689 6417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:13.613274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:60059 (local address 127.0.0.1:41743)
0806 06:35:13.613453 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:13.613460 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:13.613483 (+ 23us) server_negotiation.cc:407] Connection header received
0806 06:35:13.614080 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:13.614086 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:13.614131 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:13.614246 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:13.614480 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:13.615320 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:13.616099 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:13.616511 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:13.616604 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:13.617166 (+ 562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:13.617169 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:13.617170 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:35:13.617315 (+ 145us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:13.617342 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:35:13.617352 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:13.617417 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:13.617507 (+ 90us) server_negotiation.cc:299] Negotiation successful
0806 06:35:13.617532 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":81,"threads_started":1}
I20260806 06:35:13.618592 7341 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:35:13.618911 6409 raft_consensus.cc:348] T 00000000000000000000000000000000 P 18f40e9ddffd474cbef80cbbc605c02e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18f40e9ddffd474cbef80cbbc605c02e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41743 } }
I20260806 06:35:13.619030 6409 raft_consensus.cc:374] T 00000000000000000000000000000000 P 18f40e9ddffd474cbef80cbbc605c02e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:35:13.619062 6409 raft_consensus.cc:729] T 00000000000000000000000000000000 P 18f40e9ddffd474cbef80cbbc605c02e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18f40e9ddffd474cbef80cbbc605c02e, State: Initialized, Role: FOLLOWER
I20260806 06:35:13.619164 6409 consensus_queue.cc:260] T 00000000000000000000000000000000 P 18f40e9ddffd474cbef80cbbc605c02e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18f40e9ddffd474cbef80cbbc605c02e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41743 } }
I20260806 06:35:13.619218 6409 raft_consensus.cc:388] T 00000000000000000000000000000000 P 18f40e9ddffd474cbef80cbbc605c02e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:35:13.619246 6409 raft_consensus.cc:482] T 00000000000000000000000000000000 P 18f40e9ddffd474cbef80cbbc605c02e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:35:13.619290 6409 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 18f40e9ddffd474cbef80cbbc605c02e [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:35:13.619923 6409 raft_consensus.cc:504] T 00000000000000000000000000000000 P 18f40e9ddffd474cbef80cbbc605c02e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18f40e9ddffd474cbef80cbbc605c02e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41743 } }
I20260806 06:35:13.620025 6409 leader_election.cc:302] T 00000000000000000000000000000000 P 18f40e9ddffd474cbef80cbbc605c02e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 18f40e9ddffd474cbef80cbbc605c02e; no voters:
I20260806 06:35:13.620164 6409 leader_election.cc:288] T 00000000000000000000000000000000 P 18f40e9ddffd474cbef80cbbc605c02e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:35:13.620194 6427 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 18f40e9ddffd474cbef80cbbc605c02e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:35:13.620355 6427 raft_consensus.cc:686] T 00000000000000000000000000000000 P 18f40e9ddffd474cbef80cbbc605c02e [term 1 LEADER]: Becoming Leader. State: Replica: 18f40e9ddffd474cbef80cbbc605c02e, State: Running, Role: LEADER
I20260806 06:35:13.620402 6409 sys_catalog.cc:564] T 00000000000000000000000000000000 P 18f40e9ddffd474cbef80cbbc605c02e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:35:13.620460 6427 consensus_queue.cc:237] T 00000000000000000000000000000000 P 18f40e9ddffd474cbef80cbbc605c02e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18f40e9ddffd474cbef80cbbc605c02e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41743 } }
I20260806 06:35:13.620781 6427 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18f40e9ddffd474cbef80cbbc605c02e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 18f40e9ddffd474cbef80cbbc605c02e. Latest consensus state: current_term: 1 leader_uuid: "18f40e9ddffd474cbef80cbbc605c02e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18f40e9ddffd474cbef80cbbc605c02e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41743 } } }
I20260806 06:35:13.621014 6427 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18f40e9ddffd474cbef80cbbc605c02e [sys.catalog]: This master's current role is: LEADER
I20260806 06:35:13.620951 6428 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18f40e9ddffd474cbef80cbbc605c02e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "18f40e9ddffd474cbef80cbbc605c02e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18f40e9ddffd474cbef80cbbc605c02e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41743 } } }
I20260806 06:35:13.621402 6428 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18f40e9ddffd474cbef80cbbc605c02e [sys.catalog]: This master's current role is: LEADER
I20260806 06:35:13.621621 6438 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:35:13.622162 6438 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:35:13.622999 6417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:13.619826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:60567 (local address 127.0.0.1:41743)
0806 06:35:13.619857 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:13.619858 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:13.619972 (+ 114us) server_negotiation.cc:407] Connection header received
0806 06:35:13.620056 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:13.620057 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:13.620065 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:13.620112 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:13.620274 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:13.620872 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:13.621824 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:13.622091 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:13.622155 (+ 64us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:13.622786 (+ 631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:13.622787 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:13.622787 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:35:13.622791 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:13.622814 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:35:13.622817 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:13.622862 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:13.622928 (+ 66us) server_negotiation.cc:299] Negotiation successful
0806 06:35:13.622936 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260806 06:35:13.623548 6438 catalog_manager.cc:1347] Generated new cluster ID: e24a75ac193048e6a5c9b0d64396f3b9
I20260806 06:35:13.623592 6438 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:35:13.648924 6438 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:35:13.649598 6438 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:35:13.661684 6438 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 18f40e9ddffd474cbef80cbbc605c02e: Generated new TSK 0
I20260806 06:35:13.661845 6438 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:35:13.670040 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 6306
W20260806 06:35:13.680243 6287 connection.cc:535] client connection to 127.0.0.1:41743 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41743): Connection reset by peer (error 104)
2026-08-06T06:35:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1382 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785997861828271-7341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:35:13 dist-test-slave-6zbq krb5kdc[6454](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:35:13 dist-test-slave-6zbq krb5kdc[6454](info): set up 2 sockets
Aug 06 06:35:13 dist-test-slave-6zbq krb5kdc[6454](info): commencing operation
krb5kdc: starting...
W20260806 06:35:14.781652 7341 mini_kdc.cc:121] Time spent starting KDC: real 1.088s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Aug 06 06:35:14 dist-test-slave-6zbq krb5kdc[6454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998114, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:35:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:35:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785997861828271-7341-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785997861828271-7341-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:35:14.886499 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45097
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785997861828271-7341-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45097
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:35:14.964900 6468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:35:14.965075 6468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:35:14.965096 6468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:35:14.966161 6468 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:35:14.966195 6468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:35:14.966208 6468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:35:14.966219 6468 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:35:14.966233 6468 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:35:14.967618 6468 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785997861828271-7341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45097
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785997861828271-7341-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6468
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:35:14.967796 6468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:35:14.967942 6468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:35:14.972347 6474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:14.972370 6475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:14.972489 6468 server_base.cc:1034] running on GCE node
W20260806 06:35:14.972348 6477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:14.972785 6468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:35:14.973047 6468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:35:14.974217 6468 hybrid_clock.cc:648] HybridClock initialized: now 1785998114974196 us; error 31 us; skew 500 ppm
Aug 06 06:35:14 dist-test-slave-6zbq krb5kdc[6454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998114, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:35:14.976404 6468 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:35:14.976770 6468 webserver.cc:460] Webserver started at http://127.0.0.1:40259/ using document root <none> and password file <none>
I20260806 06:35:14.976948 6468 fs_manager.cc:359] Metadata directory not provided
I20260806 06:35:14.976992 6468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:35:14.977111 6468 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:35:14.978029 6468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785997861828271-7341-0/minicluster-data/master-0/data/instance:
uuid: "28c78f3180914e94a3ee48759072a3c4"
format_stamp: "Formatted at 2026-08-06 06:35:14 on dist-test-slave-6zbq"
server_key: "946066cc7accac6df8647f22f8b3d1ee"
server_key_iv: "876e4fd11c0a8488a60cc4b854de50cb"
server_key_version: "encryptionkey@0"
I20260806 06:35:14.978363 6468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785997861828271-7341-0/minicluster-data/master-0/wal/instance:
uuid: "28c78f3180914e94a3ee48759072a3c4"
format_stamp: "Formatted at 2026-08-06 06:35:14 on dist-test-slave-6zbq"
server_key: "946066cc7accac6df8647f22f8b3d1ee"
server_key_iv: "876e4fd11c0a8488a60cc4b854de50cb"
server_key_version: "encryptionkey@0"
I20260806 06:35:14.979512 6468 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:35:14.980129 6488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:14.980280 6468 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:35:14.980361 6468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "28c78f3180914e94a3ee48759072a3c4"
format_stamp: "Formatted at 2026-08-06 06:35:14 on dist-test-slave-6zbq"
server_key: "946066cc7accac6df8647f22f8b3d1ee"
server_key_iv: "876e4fd11c0a8488a60cc4b854de50cb"
server_key_version: "encryptionkey@0"
I20260806 06:35:14.980424 6468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:35:14.983484 6468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:35:14.984989 6468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:35:14.985108 6468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:35:14.988583 6468 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45097
I20260806 06:35:14.988612 6563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45097 every 8 connection(s)
I20260806 06:35:14.988940 6468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:35:14.989657 6565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:35:14.990159 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 6468
I20260806 06:35:14.990275 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785997861828271-7341-0/minicluster-data/master-0/wal/instance
I20260806 06:35:14.990465 7341 external_mini_cluster.cc:1428] Setting key be4a4ce650e68647d24e5508d299fbc4
I20260806 06:35:14.992219 6565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28c78f3180914e94a3ee48759072a3c4: Bootstrap starting.
I20260806 06:35:14.992544 6569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:14.990894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:40309 (local address 127.0.0.1:45097)
0806 06:35:14.991037 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:14.991044 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:14.991066 (+ 22us) server_negotiation.cc:407] Connection header received
0806 06:35:14.991435 (+ 369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:14.991441 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:14.991470 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:14.991572 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:14.992120 (+ 548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:14.992124 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:14.992124 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:35:14.992127 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:14.992175 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:35:14.992184 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:14.992242 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:14.992334 (+ 92us) server_negotiation.cc:299] Negotiation successful
0806 06:35:14.992377 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":57,"threads_started":1}
I20260806 06:35:14.992847 6565 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 28c78f3180914e94a3ee48759072a3c4: Neither blocks nor log segments found. Creating new log.
I20260806 06:35:14.993129 6565 log.cc:824] T 00000000000000000000000000000000 P 28c78f3180914e94a3ee48759072a3c4: Log is configured to *not* fsync() on all Append() calls
I20260806 06:35:14.993407 7341 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:35:14.993752 6565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28c78f3180914e94a3ee48759072a3c4: No bootstrap required, opened a new log
I20260806 06:35:14.995572 6569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:14.994661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:55033 (local address 127.0.0.1:45097)
0806 06:35:14.994705 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:14.994707 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:14.994822 (+ 115us) server_negotiation.cc:407] Connection header received
0806 06:35:14.994872 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:14.994873 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:14.994877 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:14.994933 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:14.995385 (+ 452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:14.995386 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:14.995387 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:35:14.995391 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:14.995405 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:35:14.995409 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:14.995465 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:14.995502 (+ 37us) server_negotiation.cc:299] Negotiation successful
0806 06:35:14.995514 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 06:35:14.995910 6565 raft_consensus.cc:348] T 00000000000000000000000000000000 P 28c78f3180914e94a3ee48759072a3c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28c78f3180914e94a3ee48759072a3c4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45097 } }
I20260806 06:35:14.996060 6565 raft_consensus.cc:374] T 00000000000000000000000000000000 P 28c78f3180914e94a3ee48759072a3c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:35:14.996098 6565 raft_consensus.cc:729] T 00000000000000000000000000000000 P 28c78f3180914e94a3ee48759072a3c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28c78f3180914e94a3ee48759072a3c4, State: Initialized, Role: FOLLOWER
I20260806 06:35:14.996217 6565 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28c78f3180914e94a3ee48759072a3c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28c78f3180914e94a3ee48759072a3c4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45097 } }
I20260806 06:35:14.996291 6565 raft_consensus.cc:388] T 00000000000000000000000000000000 P 28c78f3180914e94a3ee48759072a3c4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:35:14.996323 6565 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28c78f3180914e94a3ee48759072a3c4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:35:14.996368 6565 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28c78f3180914e94a3ee48759072a3c4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:35:14.996833 6565 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28c78f3180914e94a3ee48759072a3c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28c78f3180914e94a3ee48759072a3c4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45097 } }
I20260806 06:35:14.996920 6565 leader_election.cc:302] T 00000000000000000000000000000000 P 28c78f3180914e94a3ee48759072a3c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 28c78f3180914e94a3ee48759072a3c4; no voters:
I20260806 06:35:14.997068 6565 leader_election.cc:288] T 00000000000000000000000000000000 P 28c78f3180914e94a3ee48759072a3c4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:35:14.997112 6579 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 28c78f3180914e94a3ee48759072a3c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:35:14.997255 6579 raft_consensus.cc:686] T 00000000000000000000000000000000 P 28c78f3180914e94a3ee48759072a3c4 [term 1 LEADER]: Becoming Leader. State: Replica: 28c78f3180914e94a3ee48759072a3c4, State: Running, Role: LEADER
I20260806 06:35:14.997290 6565 sys_catalog.cc:564] T 00000000000000000000000000000000 P 28c78f3180914e94a3ee48759072a3c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:35:14.997339 6579 consensus_queue.cc:237] T 00000000000000000000000000000000 P 28c78f3180914e94a3ee48759072a3c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28c78f3180914e94a3ee48759072a3c4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45097 } }
I20260806 06:35:14.997678 6581 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28c78f3180914e94a3ee48759072a3c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "28c78f3180914e94a3ee48759072a3c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28c78f3180914e94a3ee48759072a3c4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45097 } } }
I20260806 06:35:14.997682 6584 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28c78f3180914e94a3ee48759072a3c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28c78f3180914e94a3ee48759072a3c4. Latest consensus state: current_term: 1 leader_uuid: "28c78f3180914e94a3ee48759072a3c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28c78f3180914e94a3ee48759072a3c4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45097 } } }
I20260806 06:35:14.997856 6584 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28c78f3180914e94a3ee48759072a3c4 [sys.catalog]: This master's current role is: LEADER
I20260806 06:35:14.998032 6581 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28c78f3180914e94a3ee48759072a3c4 [sys.catalog]: This master's current role is: LEADER
I20260806 06:35:14.998196 6596 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:35:14.998654 6596 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:35:15.000003 6596 catalog_manager.cc:1347] Generated new cluster ID: cfaca48466774f18807b5718b81f4a66
I20260806 06:35:15.000054 6596 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:35:15.013103 6596 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:35:15.013232 6596 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:35:15.017740 6596 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 28c78f3180914e94a3ee48759072a3c4: Generated new TSK 0
I20260806 06:35:15.017880 6596 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:35:15.020244 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 6468
2026-08-06T06:35:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1355 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785997861828271-7341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:35:15 dist-test-slave-6zbq krb5kdc[6613](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:35:15 dist-test-slave-6zbq krb5kdc[6613](info): set up 2 sockets
Aug 06 06:35:15 dist-test-slave-6zbq krb5kdc[6613](info): commencing operation
krb5kdc: starting...
W20260806 06:35:16.130965 7341 mini_kdc.cc:121] Time spent starting KDC: real 1.082s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Aug 06 06:35:16 dist-test-slave-6zbq krb5kdc[6613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998116, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:35:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:35:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.43.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
I20260806 06:35:16.239004 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:41017
--webserver_interface=127.7.43.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:37247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.43.126:41017
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:35:16.316769 6633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:35:16.316943 6633 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:35:16.316974 6633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:35:16.318054 6633 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:35:16.318094 6633 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:35:16.318111 6633 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:35:16.318141 6633 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:35:16.318166 6633 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:35:16.319588 6633 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:37247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785997861828271-7341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.43.126:41017
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:41017
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.43.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6633
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:35:16.319805 6633 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:35:16.319991 6633 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:35:16.324426 6644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:16.324460 6641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:16.324483 6642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:16.324502 6633 server_base.cc:1034] running on GCE node
I20260806 06:35:16.324788 6633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:35:16.325031 6633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:35:16.326164 6633 hybrid_clock.cc:648] HybridClock initialized: now 1785998116326147 us; error 28 us; skew 500 ppm
Aug 06 06:35:16 dist-test-slave-6zbq krb5kdc[6613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998116, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:35:16.333014 6633 init.cc:377] Logged in from keytab as kudu/127.7.43.126@KRBTEST.COM (short username kudu)
I20260806 06:35:16.333418 6633 webserver.cc:460] Webserver started at http://127.7.43.126:38999/ using document root <none> and password file <none>
I20260806 06:35:16.333597 6633 fs_manager.cc:359] Metadata directory not provided
I20260806 06:35:16.333645 6633 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:35:16.333760 6633 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:35:16.334713 6633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785997861828271-7341-0/minicluster-data/master-0/data/instance:
uuid: "27f7e76c975c4a9889a44fedff81b773"
format_stamp: "Formatted at 2026-08-06 06:35:16 on dist-test-slave-6zbq"
server_key: "ad44b1a229e27915236a53ff8365e2a6"
server_key_iv: "a1d79220f2b279103dd8326f5e48fcea"
server_key_version: "encryptionkey@0"
I20260806 06:35:16.335012 6633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785997861828271-7341-0/minicluster-data/master-0/wal/instance:
uuid: "27f7e76c975c4a9889a44fedff81b773"
format_stamp: "Formatted at 2026-08-06 06:35:16 on dist-test-slave-6zbq"
server_key: "ad44b1a229e27915236a53ff8365e2a6"
server_key_iv: "a1d79220f2b279103dd8326f5e48fcea"
server_key_version: "encryptionkey@0"
I20260806 06:35:16.336234 6633 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:35:16.336830 6656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:16.336980 6633 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:35:16.337067 6633 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "27f7e76c975c4a9889a44fedff81b773"
format_stamp: "Formatted at 2026-08-06 06:35:16 on dist-test-slave-6zbq"
server_key: "ad44b1a229e27915236a53ff8365e2a6"
server_key_iv: "a1d79220f2b279103dd8326f5e48fcea"
server_key_version: "encryptionkey@0"
I20260806 06:35:16.337134 6633 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:35:16.350102 6633 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:35:16.351644 6633 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:35:16.351760 6633 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:35:16.355333 6633 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.126:41017
I20260806 06:35:16.355362 6731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.126:41017 every 8 connection(s)
I20260806 06:35:16.355638 6633 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:35:16.356367 6732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:35:16.358314 6732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27f7e76c975c4a9889a44fedff81b773: Bootstrap starting.
I20260806 06:35:16.358844 6732 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 27f7e76c975c4a9889a44fedff81b773: Neither blocks nor log segments found. Creating new log.
I20260806 06:35:16.359090 6732 log.cc:824] T 00000000000000000000000000000000 P 27f7e76c975c4a9889a44fedff81b773: Log is configured to *not* fsync() on all Append() calls
I20260806 06:35:16.359684 6732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27f7e76c975c4a9889a44fedff81b773: No bootstrap required, opened a new log
I20260806 06:35:16.361166 6732 raft_consensus.cc:348] T 00000000000000000000000000000000 P 27f7e76c975c4a9889a44fedff81b773 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f7e76c975c4a9889a44fedff81b773" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 41017 } }
I20260806 06:35:16.361295 6732 raft_consensus.cc:374] T 00000000000000000000000000000000 P 27f7e76c975c4a9889a44fedff81b773 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:35:16.361328 6732 raft_consensus.cc:729] T 00000000000000000000000000000000 P 27f7e76c975c4a9889a44fedff81b773 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27f7e76c975c4a9889a44fedff81b773, State: Initialized, Role: FOLLOWER
I20260806 06:35:16.361454 6732 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27f7e76c975c4a9889a44fedff81b773 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f7e76c975c4a9889a44fedff81b773" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 41017 } }
I20260806 06:35:16.361512 6732 raft_consensus.cc:388] T 00000000000000000000000000000000 P 27f7e76c975c4a9889a44fedff81b773 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:35:16.361533 6732 raft_consensus.cc:482] T 00000000000000000000000000000000 P 27f7e76c975c4a9889a44fedff81b773 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:35:16.361577 6732 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 27f7e76c975c4a9889a44fedff81b773 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:35:16.362323 6732 raft_consensus.cc:504] T 00000000000000000000000000000000 P 27f7e76c975c4a9889a44fedff81b773 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f7e76c975c4a9889a44fedff81b773" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 41017 } }
I20260806 06:35:16.362440 6732 leader_election.cc:302] T 00000000000000000000000000000000 P 27f7e76c975c4a9889a44fedff81b773 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27f7e76c975c4a9889a44fedff81b773; no voters:
I20260806 06:35:16.362557 6732 leader_election.cc:288] T 00000000000000000000000000000000 P 27f7e76c975c4a9889a44fedff81b773 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:35:16.362593 6736 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 27f7e76c975c4a9889a44fedff81b773 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:35:16.362788 6736 raft_consensus.cc:686] T 00000000000000000000000000000000 P 27f7e76c975c4a9889a44fedff81b773 [term 1 LEADER]: Becoming Leader. State: Replica: 27f7e76c975c4a9889a44fedff81b773, State: Running, Role: LEADER
I20260806 06:35:16.362818 6732 sys_catalog.cc:564] T 00000000000000000000000000000000 P 27f7e76c975c4a9889a44fedff81b773 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:35:16.362807 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 6633
I20260806 06:35:16.362911 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785997861828271-7341-0/minicluster-data/master-0/wal/instance
I20260806 06:35:16.362900 6736 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27f7e76c975c4a9889a44fedff81b773 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f7e76c975c4a9889a44fedff81b773" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 41017 } }
I20260806 06:35:16.363086 7341 external_mini_cluster.cc:1428] Setting key 876e9b8803c8533f094079d5a94fc88c
I20260806 06:35:16.363253 6736 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27f7e76c975c4a9889a44fedff81b773 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27f7e76c975c4a9889a44fedff81b773. Latest consensus state: current_term: 1 leader_uuid: "27f7e76c975c4a9889a44fedff81b773" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f7e76c975c4a9889a44fedff81b773" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 41017 } } }
I20260806 06:35:16.363306 6738 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27f7e76c975c4a9889a44fedff81b773 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "27f7e76c975c4a9889a44fedff81b773" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f7e76c975c4a9889a44fedff81b773" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 41017 } } }
I20260806 06:35:16.363317 6736 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27f7e76c975c4a9889a44fedff81b773 [sys.catalog]: This master's current role is: LEADER
I20260806 06:35:16.363373 6738 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27f7e76c975c4a9889a44fedff81b773 [sys.catalog]: This master's current role is: LEADER
I20260806 06:35:16.363596 6748 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:35:16.364162 6748 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:35:16.365688 6748 catalog_manager.cc:1347] Generated new cluster ID: 648a2d8f1d9e4e6ea3266f7d974709ae
I20260806 06:35:16.365751 6748 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:35:16 dist-test-slave-6zbq krb5kdc[6613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998116, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:35:16.372825 6751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:16.363684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59337 (local address 127.7.43.126:41017)
0806 06:35:16.364002 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:16.364011 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:16.364562 (+ 551us) server_negotiation.cc:407] Connection header received
0806 06:35:16.364735 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:16.364741 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:16.364791 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:16.364923 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:16.365644 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:16.366637 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:16.367367 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:16.367784 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:16.367875 (+ 91us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:16.369939 (+ 2064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:16.369944 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:16.369944 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:16.369954 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:16.371428 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:16.371788 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:16.371791 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:16.371791 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:16.371879 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:16.372080 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:16.372081 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:16.372081 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:16.372307 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:16.372462 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:16.372581 (+ 119us) server_negotiation.cc:299] Negotiation successful
0806 06:35:16.372676 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":81,"threads_started":1}
I20260806 06:35:16.373646 7341 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:35:16.380826 6748 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:35:16.381482 6748 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:35:16.384049 6751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:16.378561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45747 (local address 127.7.43.126:41017)
0806 06:35:16.378601 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:16.378603 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:16.378701 (+ 98us) server_negotiation.cc:407] Connection header received
0806 06:35:16.378754 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:16.378755 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:16.378774 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:16.378817 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:16.379291 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:16.379791 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:16.380406 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:16.380706 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:16.380779 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:16.381855 (+ 1076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:16.381856 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:16.381856 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:16.381862 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:16.383061 (+ 1199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:16.383381 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:16.383382 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:16.383382 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:16.383435 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:16.383621 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:16.383622 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:16.383622 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:16.383719 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:16.383812 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:16.383919 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 06:35:16.383971 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":17}
I20260806 06:35:16.386624 6748 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 27f7e76c975c4a9889a44fedff81b773: Generated new TSK 0
I20260806 06:35:16.386777 6748 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:35:16.391455 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 6633
W20260806 06:35:16.408296 6610 connection.cc:535] client connection to 127.7.43.126:41017 recv error: Network error: failed to read from TLS socket (remote: 127.7.43.126:41017): Connection reset by peer (error 104)
2026-08-06T06:35:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1372 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785997861828271-7341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:35:16 dist-test-slave-6zbq krb5kdc[6776](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:35:16 dist-test-slave-6zbq krb5kdc[6776](info): set up 2 sockets
Aug 06 06:35:16 dist-test-slave-6zbq krb5kdc[6776](info): commencing operation
krb5kdc: starting...
W20260806 06:35:17.503878 7341 mini_kdc.cc:121] Time spent starting KDC: real 1.082s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Aug 06 06:35:17 dist-test-slave-6zbq krb5kdc[6776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998117, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:35:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:35:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.43.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
I20260806 06:35:17.609788 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:39633
--webserver_interface=127.7.43.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:34197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.43.126:39633
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:35:17.687107 6795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:35:17.687280 6795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:35:17.687300 6795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:35:17.688354 6795 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:35:17.688387 6795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:35:17.688400 6795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:35:17.688411 6795 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:35:17.688424 6795 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:35:17.689805 6795 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:34197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785997861828271-7341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.43.126:39633
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:39633
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.43.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6795
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:35:17.689975 6795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:35:17.690135 6795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:35:17.694394 6802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:17.694509 6795 server_base.cc:1034] running on GCE node
W20260806 06:35:17.694375 6801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:17.694389 6804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:17.694818 6795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:35:17.695073 6795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:35:17.696205 6795 hybrid_clock.cc:648] HybridClock initialized: now 1785998117696192 us; error 29 us; skew 500 ppm
Aug 06 06:35:17 dist-test-slave-6zbq krb5kdc[6776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998117, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:35:17.698444 6795 init.cc:377] Logged in from keytab as kudu/127.7.43.126@KRBTEST.COM (short username kudu)
I20260806 06:35:17.698823 6795 webserver.cc:460] Webserver started at http://127.7.43.126:33979/ using document root <none> and password file <none>
I20260806 06:35:17.699028 6795 fs_manager.cc:359] Metadata directory not provided
I20260806 06:35:17.699072 6795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:35:17.699190 6795 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:35:17.700181 6795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785997861828271-7341-0/minicluster-data/master-0/data/instance:
uuid: "8f25d7b8ab2b4c169ae794a05401c6a5"
format_stamp: "Formatted at 2026-08-06 06:35:17 on dist-test-slave-6zbq"
server_key: "796551b45f43c78e984387dcb6aecf05"
server_key_iv: "baa543dea25c256d2971a0aae803f798"
server_key_version: "encryptionkey@0"
I20260806 06:35:17.700485 6795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785997861828271-7341-0/minicluster-data/master-0/wal/instance:
uuid: "8f25d7b8ab2b4c169ae794a05401c6a5"
format_stamp: "Formatted at 2026-08-06 06:35:17 on dist-test-slave-6zbq"
server_key: "796551b45f43c78e984387dcb6aecf05"
server_key_iv: "baa543dea25c256d2971a0aae803f798"
server_key_version: "encryptionkey@0"
I20260806 06:35:17.701640 6795 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:35:17.702229 6813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:17.702409 6795 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:35:17.702491 6795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "8f25d7b8ab2b4c169ae794a05401c6a5"
format_stamp: "Formatted at 2026-08-06 06:35:17 on dist-test-slave-6zbq"
server_key: "796551b45f43c78e984387dcb6aecf05"
server_key_iv: "baa543dea25c256d2971a0aae803f798"
server_key_version: "encryptionkey@0"
I20260806 06:35:17.702557 6795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:35:17.738047 6795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:35:17.739706 6795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:35:17.739835 6795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:35:17.743511 6795 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.126:39633
I20260806 06:35:17.743557 6893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.126:39633 every 8 connection(s)
I20260806 06:35:17.743960 6795 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:35:17.744930 6894 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:35:17.747371 6894 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f25d7b8ab2b4c169ae794a05401c6a5: Bootstrap starting.
I20260806 06:35:17.747872 6894 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f25d7b8ab2b4c169ae794a05401c6a5: Neither blocks nor log segments found. Creating new log.
I20260806 06:35:17.748090 6894 log.cc:824] T 00000000000000000000000000000000 P 8f25d7b8ab2b4c169ae794a05401c6a5: Log is configured to *not* fsync() on all Append() calls
I20260806 06:35:17.748682 6894 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f25d7b8ab2b4c169ae794a05401c6a5: No bootstrap required, opened a new log
I20260806 06:35:17.750172 6894 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f25d7b8ab2b4c169ae794a05401c6a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f25d7b8ab2b4c169ae794a05401c6a5" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 39633 } }
I20260806 06:35:17.750301 6894 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f25d7b8ab2b4c169ae794a05401c6a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:35:17.750327 6894 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f25d7b8ab2b4c169ae794a05401c6a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f25d7b8ab2b4c169ae794a05401c6a5, State: Initialized, Role: FOLLOWER
I20260806 06:35:17.750470 6894 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f25d7b8ab2b4c169ae794a05401c6a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f25d7b8ab2b4c169ae794a05401c6a5" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 39633 } }
I20260806 06:35:17.750530 6894 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f25d7b8ab2b4c169ae794a05401c6a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:35:17.750550 6894 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f25d7b8ab2b4c169ae794a05401c6a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:35:17.750578 6894 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f25d7b8ab2b4c169ae794a05401c6a5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:35:17.751298 6894 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f25d7b8ab2b4c169ae794a05401c6a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f25d7b8ab2b4c169ae794a05401c6a5" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 39633 } }
I20260806 06:35:17.751384 6894 leader_election.cc:302] T 00000000000000000000000000000000 P 8f25d7b8ab2b4c169ae794a05401c6a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f25d7b8ab2b4c169ae794a05401c6a5; no voters:
I20260806 06:35:17.751495 6894 leader_election.cc:288] T 00000000000000000000000000000000 P 8f25d7b8ab2b4c169ae794a05401c6a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:35:17.751533 6898 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f25d7b8ab2b4c169ae794a05401c6a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:35:17.751675 6898 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f25d7b8ab2b4c169ae794a05401c6a5 [term 1 LEADER]: Becoming Leader. State: Replica: 8f25d7b8ab2b4c169ae794a05401c6a5, State: Running, Role: LEADER
I20260806 06:35:17.751730 6894 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f25d7b8ab2b4c169ae794a05401c6a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:35:17.751814 6898 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f25d7b8ab2b4c169ae794a05401c6a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f25d7b8ab2b4c169ae794a05401c6a5" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 39633 } }
I20260806 06:35:17.752177 6903 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f25d7b8ab2b4c169ae794a05401c6a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f25d7b8ab2b4c169ae794a05401c6a5. Latest consensus state: current_term: 1 leader_uuid: "8f25d7b8ab2b4c169ae794a05401c6a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f25d7b8ab2b4c169ae794a05401c6a5" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 39633 } } }
I20260806 06:35:17.752185 6902 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f25d7b8ab2b4c169ae794a05401c6a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f25d7b8ab2b4c169ae794a05401c6a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f25d7b8ab2b4c169ae794a05401c6a5" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 39633 } } }
I20260806 06:35:17.752339 6903 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f25d7b8ab2b4c169ae794a05401c6a5 [sys.catalog]: This master's current role is: LEADER
I20260806 06:35:17.752353 6902 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f25d7b8ab2b4c169ae794a05401c6a5 [sys.catalog]: This master's current role is: LEADER
I20260806 06:35:17.752771 6913 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:35:17.753232 6913 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:35:17.753944 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 6795
I20260806 06:35:17.754072 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785997861828271-7341-0/minicluster-data/master-0/wal/instance
I20260806 06:35:17.754251 7341 external_mini_cluster.cc:1428] Setting key 534f7b9e7569eda4b269adf69c84e52f
I20260806 06:35:17.754751 6913 catalog_manager.cc:1347] Generated new cluster ID: 405f958f64c8497ebe16efed5bb9a1bd
I20260806 06:35:17.754804 6913 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:35:17.762801 6928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:17.754664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33457 (local address 127.7.43.126:39633)
0806 06:35:17.754800 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:17.754809 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:17.758625 (+ 3816us) server_negotiation.cc:407] Connection header received
0806 06:35:17.758746 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:17.758752 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:17.758803 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:17.758908 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:17.759102 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:17.760339 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:17.761076 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:17.761492 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:17.761584 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:17.762171 (+ 587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:17.762175 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:17.762176 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:35:17.762331 (+ 155us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:17.762389 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:35:17.762404 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:17.762468 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:17.762565 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 06:35:17.762592 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":77,"threads_started":1}
I20260806 06:35:17.763617 7341 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:35:17.765511 6913 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:35:17.766149 6913 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:35:17.768693 6913 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f25d7b8ab2b4c169ae794a05401c6a5: Generated new TSK 0
I20260806 06:35:17.768854 6913 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:35:17.769064 6928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:17.765405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45841 (local address 127.7.43.126:39633)
0806 06:35:17.765438 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:17.765439 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:17.766581 (+ 1142us) server_negotiation.cc:407] Connection header received
0806 06:35:17.766641 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:17.766642 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:17.766651 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:17.766712 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:17.766832 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:17.767310 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:17.767977 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:17.768179 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:17.768229 (+ 50us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:17.768851 (+ 622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:17.768852 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:17.768852 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:35:17.768856 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:17.768866 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:35:17.768870 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:17.768913 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:17.768982 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 06:35:17.768992 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":11,"server-negotiator.queue_time_us":15}
I20260806 06:35:17.770681 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 6795
W20260806 06:35:17.787370 6773 connection.cc:535] client connection to 127.7.43.126:39633 recv error: Network error: failed to read from TLS socket (remote: 127.7.43.126:39633): Connection reset by peer (error 104)
2026-08-06T06:35:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1379 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785997861828271-7341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:35:17 dist-test-slave-6zbq krb5kdc[6942](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:35:17 dist-test-slave-6zbq krb5kdc[6942](info): set up 2 sockets
Aug 06 06:35:17 dist-test-slave-6zbq krb5kdc[6942](info): commencing operation
krb5kdc: starting...
W20260806 06:35:18.875566 7341 mini_kdc.cc:121] Time spent starting KDC: real 1.075s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Aug 06 06:35:18 dist-test-slave-6zbq krb5kdc[6942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998118, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:35:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:35:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.43.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
I20260806 06:35:18.982928 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:34105
--webserver_interface=127.7.43.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:38257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.43.126:34105
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:35:19.071528 6960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:35:19.071704 6960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:35:19.071733 6960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:35:19.072815 6960 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:35:19.072853 6960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:35:19.072873 6960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:35:19.072889 6960 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:35:19.072906 6960 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:35:19.074323 6960 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:38257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785997861828271-7341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.43.126:34105
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:34105
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.43.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6960
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:35:19.074556 6960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:35:19.074769 6960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:35:19.079044 6967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:19.079108 6966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:19.079052 6969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:19.079273 6960 server_base.cc:1034] running on GCE node
I20260806 06:35:19.079494 6960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:35:19.079739 6960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:35:19.080881 6960 hybrid_clock.cc:648] HybridClock initialized: now 1785998119080861 us; error 33 us; skew 500 ppm
Aug 06 06:35:19 dist-test-slave-6zbq krb5kdc[6942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998119, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:35:19.083108 6960 init.cc:377] Logged in from keytab as kudu/127.7.43.126@KRBTEST.COM (short username kudu)
I20260806 06:35:19.083469 6960 webserver.cc:460] Webserver started at http://127.7.43.126:35407/ using document root <none> and password file <none>
I20260806 06:35:19.083650 6960 fs_manager.cc:359] Metadata directory not provided
I20260806 06:35:19.083688 6960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:35:19.083770 6960 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:35:19.084667 6960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785997861828271-7341-0/minicluster-data/master-0/data/instance:
uuid: "51f8d94a61414c52ac933461230e2455"
format_stamp: "Formatted at 2026-08-06 06:35:19 on dist-test-slave-6zbq"
server_key: "ae4e5980bc05a2572544823179fac4c7"
server_key_iv: "40d8b095f57b987f56c499ec80b197e1"
server_key_version: "encryptionkey@0"
I20260806 06:35:19.084944 6960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785997861828271-7341-0/minicluster-data/master-0/wal/instance:
uuid: "51f8d94a61414c52ac933461230e2455"
format_stamp: "Formatted at 2026-08-06 06:35:19 on dist-test-slave-6zbq"
server_key: "ae4e5980bc05a2572544823179fac4c7"
server_key_iv: "40d8b095f57b987f56c499ec80b197e1"
server_key_version: "encryptionkey@0"
I20260806 06:35:19.086078 6960 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:35:19.086817 6977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:19.086956 6960 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:19.087028 6960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "51f8d94a61414c52ac933461230e2455"
format_stamp: "Formatted at 2026-08-06 06:35:19 on dist-test-slave-6zbq"
server_key: "ae4e5980bc05a2572544823179fac4c7"
server_key_iv: "40d8b095f57b987f56c499ec80b197e1"
server_key_version: "encryptionkey@0"
I20260806 06:35:19.087077 6960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:35:19.090027 6960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:35:19.091611 6960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:35:19.091717 6960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:35:19.095157 6960 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.126:34105
I20260806 06:35:19.095198 7053 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.126:34105 every 8 connection(s)
I20260806 06:35:19.095449 6960 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:35:19.096315 7055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:35:19.097028 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 6960
I20260806 06:35:19.097123 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785997861828271-7341-0/minicluster-data/master-0/wal/instance
I20260806 06:35:19.097267 7341 external_mini_cluster.cc:1428] Setting key 846473aa962f887d0f6ea81b53d0eeed
I20260806 06:35:19.099119 7055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51f8d94a61414c52ac933461230e2455: Bootstrap starting.
I20260806 06:35:19.099335 7058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:19.097674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51427 (local address 127.7.43.126:34105)
0806 06:35:19.097831 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:19.097839 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:19.097887 (+ 48us) server_negotiation.cc:407] Connection header received
0806 06:35:19.098298 (+ 411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:19.098307 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:19.098360 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:19.098503 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:19.098973 (+ 470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:19.098976 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:19.098977 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:35:19.098980 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:19.099019 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:35:19.099032 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:19.099103 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:19.099140 (+ 37us) server_negotiation.cc:299] Negotiation successful
0806 06:35:19.099177 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":72,"threads_started":1}
I20260806 06:35:19.099711 7055 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 51f8d94a61414c52ac933461230e2455: Neither blocks nor log segments found. Creating new log.
I20260806 06:35:19.099989 7055 log.cc:824] T 00000000000000000000000000000000 P 51f8d94a61414c52ac933461230e2455: Log is configured to *not* fsync() on all Append() calls
I20260806 06:35:19.100221 7341 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:35:19.100575 7055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51f8d94a61414c52ac933461230e2455: No bootstrap required, opened a new log
I20260806 06:35:19.102320 7055 raft_consensus.cc:348] T 00000000000000000000000000000000 P 51f8d94a61414c52ac933461230e2455 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51f8d94a61414c52ac933461230e2455" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 34105 } }
I20260806 06:35:19.102501 7055 raft_consensus.cc:374] T 00000000000000000000000000000000 P 51f8d94a61414c52ac933461230e2455 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:35:19.102551 7055 raft_consensus.cc:729] T 00000000000000000000000000000000 P 51f8d94a61414c52ac933461230e2455 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51f8d94a61414c52ac933461230e2455, State: Initialized, Role: FOLLOWER
I20260806 06:35:19.102671 7055 consensus_queue.cc:260] T 00000000000000000000000000000000 P 51f8d94a61414c52ac933461230e2455 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51f8d94a61414c52ac933461230e2455" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 34105 } }
I20260806 06:35:19.102774 7055 raft_consensus.cc:388] T 00000000000000000000000000000000 P 51f8d94a61414c52ac933461230e2455 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:35:19.102806 7055 raft_consensus.cc:482] T 00000000000000000000000000000000 P 51f8d94a61414c52ac933461230e2455 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:35:19.102854 7055 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 51f8d94a61414c52ac933461230e2455 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:35:19.103325 7055 raft_consensus.cc:504] T 00000000000000000000000000000000 P 51f8d94a61414c52ac933461230e2455 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51f8d94a61414c52ac933461230e2455" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 34105 } }
I20260806 06:35:19.103416 7055 leader_election.cc:302] T 00000000000000000000000000000000 P 51f8d94a61414c52ac933461230e2455 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 51f8d94a61414c52ac933461230e2455; no voters:
I20260806 06:35:19.103557 7055 leader_election.cc:288] T 00000000000000000000000000000000 P 51f8d94a61414c52ac933461230e2455 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:35:19.103621 7070 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 51f8d94a61414c52ac933461230e2455 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:35:19.103694 7058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:19.102612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48421 (local address 127.7.43.126:34105)
0806 06:35:19.102649 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:19.102651 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:19.102819 (+ 168us) server_negotiation.cc:407] Connection header received
0806 06:35:19.102897 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:19.102898 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:19.102902 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:19.102949 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:19.103472 (+ 523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:19.103473 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:19.103473 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:35:19.103477 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:19.103489 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:35:19.103493 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:19.103521 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:19.103626 (+ 105us) server_negotiation.cc:299] Negotiation successful
0806 06:35:19.103639 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 06:35:19.103786 7055 sys_catalog.cc:564] T 00000000000000000000000000000000 P 51f8d94a61414c52ac933461230e2455 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:35:19.103857 7070 raft_consensus.cc:686] T 00000000000000000000000000000000 P 51f8d94a61414c52ac933461230e2455 [term 1 LEADER]: Becoming Leader. State: Replica: 51f8d94a61414c52ac933461230e2455, State: Running, Role: LEADER
I20260806 06:35:19.103940 7070 consensus_queue.cc:237] T 00000000000000000000000000000000 P 51f8d94a61414c52ac933461230e2455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51f8d94a61414c52ac933461230e2455" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 34105 } }
I20260806 06:35:19.104240 7070 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51f8d94a61414c52ac933461230e2455 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "51f8d94a61414c52ac933461230e2455" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51f8d94a61414c52ac933461230e2455" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 34105 } } }
I20260806 06:35:19.104316 7070 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51f8d94a61414c52ac933461230e2455 [sys.catalog]: This master's current role is: LEADER
I20260806 06:35:19.104437 7070 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51f8d94a61414c52ac933461230e2455 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 51f8d94a61414c52ac933461230e2455. Latest consensus state: current_term: 1 leader_uuid: "51f8d94a61414c52ac933461230e2455" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51f8d94a61414c52ac933461230e2455" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 34105 } } }
I20260806 06:35:19.104494 7070 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51f8d94a61414c52ac933461230e2455 [sys.catalog]: This master's current role is: LEADER
I20260806 06:35:19.104571 7079 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:35:19.105011 7079 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:35:19.106426 7079 catalog_manager.cc:1347] Generated new cluster ID: 2670b18a3d284a9b8b509843f7c41394
I20260806 06:35:19.106474 7079 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:35:19.118113 7079 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:35:19.118247 7079 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:35:19.122869 7079 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 51f8d94a61414c52ac933461230e2455: Generated new TSK 0
I20260806 06:35:19.123008 7079 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:35:19.128400 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 6960
2026-08-06T06:35:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1351 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785997861828271-7341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:35:19 dist-test-slave-6zbq krb5kdc[7097](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:35:19 dist-test-slave-6zbq krb5kdc[7097](info): set up 2 sockets
Aug 06 06:35:19 dist-test-slave-6zbq krb5kdc[7097](info): commencing operation
krb5kdc: starting...
W20260806 06:35:20.222174 7341 mini_kdc.cc:121] Time spent starting KDC: real 1.070s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Aug 06 06:35:20 dist-test-slave-6zbq krb5kdc[7097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998120, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:35:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:35:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.43.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
I20260806 06:35:20.330039 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:43139
--webserver_interface=127.7.43.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:42847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.43.126:43139
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:35:20.407270 7112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:35:20.407438 7112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:35:20.407459 7112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:35:20.408484 7112 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:35:20.408516 7112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:35:20.408529 7112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:35:20.408540 7112 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:35:20.408555 7112 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:35:20.409907 7112 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:42847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785997861828271-7341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.43.126:43139
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:43139
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.43.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7112
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:35:20.410064 7112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:35:20.410223 7112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:35:20.414558 7123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:20.414619 7119 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:20.414631 7120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:20.414745 7112 server_base.cc:1034] running on GCE node
I20260806 06:35:20.414958 7112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:35:20.415200 7112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:35:20.416335 7112 hybrid_clock.cc:648] HybridClock initialized: now 1785998120416323 us; error 27 us; skew 500 ppm
Aug 06 06:35:20 dist-test-slave-6zbq krb5kdc[7097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998120, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:35:20.418504 7112 init.cc:377] Logged in from keytab as kudu/127.7.43.126@KRBTEST.COM (short username kudu)
I20260806 06:35:20.418881 7112 webserver.cc:460] Webserver started at http://127.7.43.126:34347/ using document root <none> and password file <none>
I20260806 06:35:20.419088 7112 fs_manager.cc:359] Metadata directory not provided
I20260806 06:35:20.419134 7112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:35:20.419255 7112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:35:20.420182 7112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785997861828271-7341-0/minicluster-data/master-0/data/instance:
uuid: "74928f8cc0d147c690f0e26b0c3e1241"
format_stamp: "Formatted at 2026-08-06 06:35:20 on dist-test-slave-6zbq"
server_key: "bc57d8296f9f0b05971811915835d702"
server_key_iv: "61d90ee71ead359bd697f03c377b97e2"
server_key_version: "encryptionkey@0"
I20260806 06:35:20.420493 7112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785997861828271-7341-0/minicluster-data/master-0/wal/instance:
uuid: "74928f8cc0d147c690f0e26b0c3e1241"
format_stamp: "Formatted at 2026-08-06 06:35:20 on dist-test-slave-6zbq"
server_key: "bc57d8296f9f0b05971811915835d702"
server_key_iv: "61d90ee71ead359bd697f03c377b97e2"
server_key_version: "encryptionkey@0"
I20260806 06:35:20.421638 7112 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:35:20.422248 7131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:20.422410 7112 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:35:20.422489 7112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "74928f8cc0d147c690f0e26b0c3e1241"
format_stamp: "Formatted at 2026-08-06 06:35:20 on dist-test-slave-6zbq"
server_key: "bc57d8296f9f0b05971811915835d702"
server_key_iv: "61d90ee71ead359bd697f03c377b97e2"
server_key_version: "encryptionkey@0"
I20260806 06:35:20.422554 7112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:35:20.440933 7112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:35:20.442459 7112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:35:20.442603 7112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:35:20.446031 7112 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.126:43139
I20260806 06:35:20.446079 7211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.126:43139 every 8 connection(s)
I20260806 06:35:20.446368 7112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:35:20.447105 7213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:35:20.449043 7213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74928f8cc0d147c690f0e26b0c3e1241: Bootstrap starting.
I20260806 06:35:20.449559 7213 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 74928f8cc0d147c690f0e26b0c3e1241: Neither blocks nor log segments found. Creating new log.
I20260806 06:35:20.449805 7213 log.cc:824] T 00000000000000000000000000000000 P 74928f8cc0d147c690f0e26b0c3e1241: Log is configured to *not* fsync() on all Append() calls
I20260806 06:35:20.450423 7213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74928f8cc0d147c690f0e26b0c3e1241: No bootstrap required, opened a new log
I20260806 06:35:20.451889 7213 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74928f8cc0d147c690f0e26b0c3e1241 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74928f8cc0d147c690f0e26b0c3e1241" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 43139 } }
I20260806 06:35:20.452009 7213 raft_consensus.cc:374] T 00000000000000000000000000000000 P 74928f8cc0d147c690f0e26b0c3e1241 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:35:20.452029 7213 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74928f8cc0d147c690f0e26b0c3e1241 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74928f8cc0d147c690f0e26b0c3e1241, State: Initialized, Role: FOLLOWER
I20260806 06:35:20.452157 7213 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74928f8cc0d147c690f0e26b0c3e1241 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74928f8cc0d147c690f0e26b0c3e1241" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 43139 } }
I20260806 06:35:20.452216 7213 raft_consensus.cc:388] T 00000000000000000000000000000000 P 74928f8cc0d147c690f0e26b0c3e1241 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:35:20.452236 7213 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74928f8cc0d147c690f0e26b0c3e1241 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:35:20.452278 7213 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74928f8cc0d147c690f0e26b0c3e1241 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:35:20.453011 7213 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74928f8cc0d147c690f0e26b0c3e1241 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74928f8cc0d147c690f0e26b0c3e1241" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 43139 } }
I20260806 06:35:20.453090 7213 leader_election.cc:302] T 00000000000000000000000000000000 P 74928f8cc0d147c690f0e26b0c3e1241 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 74928f8cc0d147c690f0e26b0c3e1241; no voters:
I20260806 06:35:20.453236 7213 leader_election.cc:288] T 00000000000000000000000000000000 P 74928f8cc0d147c690f0e26b0c3e1241 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:35:20.453339 7217 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74928f8cc0d147c690f0e26b0c3e1241 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:35:20.453486 7217 raft_consensus.cc:686] T 00000000000000000000000000000000 P 74928f8cc0d147c690f0e26b0c3e1241 [term 1 LEADER]: Becoming Leader. State: Replica: 74928f8cc0d147c690f0e26b0c3e1241, State: Running, Role: LEADER
I20260806 06:35:20.453502 7213 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74928f8cc0d147c690f0e26b0c3e1241 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:35:20.453639 7217 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74928f8cc0d147c690f0e26b0c3e1241 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74928f8cc0d147c690f0e26b0c3e1241" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 43139 } }
I20260806 06:35:20.453743 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 7112
I20260806 06:35:20.453871 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785997861828271-7341-0/minicluster-data/master-0/wal/instance
I20260806 06:35:20.453994 7219 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74928f8cc0d147c690f0e26b0c3e1241 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "74928f8cc0d147c690f0e26b0c3e1241" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74928f8cc0d147c690f0e26b0c3e1241" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 43139 } } }
I20260806 06:35:20.454067 7219 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74928f8cc0d147c690f0e26b0c3e1241 [sys.catalog]: This master's current role is: LEADER
I20260806 06:35:20.454087 7341 external_mini_cluster.cc:1428] Setting key 967df20345b5212fbd323bbb721ffd28
I20260806 06:35:20.454537 7221 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74928f8cc0d147c690f0e26b0c3e1241 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74928f8cc0d147c690f0e26b0c3e1241. Latest consensus state: current_term: 1 leader_uuid: "74928f8cc0d147c690f0e26b0c3e1241" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74928f8cc0d147c690f0e26b0c3e1241" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 43139 } } }
I20260806 06:35:20.454615 7221 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74928f8cc0d147c690f0e26b0c3e1241 [sys.catalog]: This master's current role is: LEADER
I20260806 06:35:20.454958 7232 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:35:20.455459 7232 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:35:20.457279 7232 catalog_manager.cc:1347] Generated new cluster ID: fac401092a494bb3b502175ab0cf61a1
I20260806 06:35:20.457316 7232 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:35:20 dist-test-slave-6zbq krb5kdc[7097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998120, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:35:20.464968 7237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:20.454747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53883 (local address 127.7.43.126:43139)
0806 06:35:20.454900 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:20.454908 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:20.454937 (+ 29us) server_negotiation.cc:407] Connection header received
0806 06:35:20.455042 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:20.455048 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:20.455099 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:20.455233 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:20.456167 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:20.457182 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:20.458217 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:20.458821 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:20.458911 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:20.461439 (+ 2528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:20.461444 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:20.461445 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:20.461457 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:20.463325 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:20.463689 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:20.463692 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:20.463693 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:20.463806 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:20.463979 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:20.463980 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:20.463980 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:20.464283 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:20.464447 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:20.464514 (+ 67us) server_negotiation.cc:299] Negotiation successful
0806 06:35:20.464760 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":80,"threads_started":1}
I20260806 06:35:20.465919 7341 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:35:20.473001 7232 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:35:20.473649 7232 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:35:20.473853 7237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:20.467278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49371 (local address 127.7.43.126:43139)
0806 06:35:20.467312 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:20.467314 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:20.467375 (+ 61us) server_negotiation.cc:407] Connection header received
0806 06:35:20.467439 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:20.467439 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:20.467449 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:20.467496 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:20.468024 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:20.468777 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:20.469654 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:20.469957 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:20.470022 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:20.471237 (+ 1215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:20.471238 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:20.471238 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:20.471245 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:20.472820 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:20.473156 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:20.473157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:20.473157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:20.473208 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:20.473372 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:20.473373 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:20.473373 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:20.473466 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:20.473553 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:20.473643 (+ 90us) server_negotiation.cc:299] Negotiation successful
0806 06:35:20.473777 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:35:20.484474 7232 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 74928f8cc0d147c690f0e26b0c3e1241: Generated new TSK 0
I20260806 06:35:20.484639 7232 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:35:20.491019 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 7112
2026-08-06T06:35:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1361 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785997861828271-7341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:35:20 dist-test-slave-6zbq krb5kdc[7261](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:35:20 dist-test-slave-6zbq krb5kdc[7261](info): set up 2 sockets
Aug 06 06:35:20 dist-test-slave-6zbq krb5kdc[7261](info): commencing operation
krb5kdc: starting...
W20260806 06:35:21.598752 7341 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Aug 06 06:35:21 dist-test-slave-6zbq krb5kdc[7261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998121, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:35:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:35:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.43.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
I20260806 06:35:21.705052 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:36715
--webserver_interface=127.7.43.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:40153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.43.126:36715
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:35:21.784415 7282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:35:21.784598 7282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:35:21.784631 7282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:35:21.785753 7282 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:35:21.785794 7282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:35:21.785813 7282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:35:21.785830 7282 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:35:21.785845 7282 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:35:21.787303 7282 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:40153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785997861828271-7341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.43.126:36715
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:36715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.43.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7282
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:35:21.787539 7282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:35:21.787730 7282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:35:21.792102 7292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:21.792137 7293 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:21.792223 7296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:21.792439 7282 server_base.cc:1034] running on GCE node
I20260806 06:35:21.792654 7282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:35:21.792918 7282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:35:21.794049 7282 hybrid_clock.cc:648] HybridClock initialized: now 1785998121794033 us; error 32 us; skew 500 ppm
Aug 06 06:35:21 dist-test-slave-6zbq krb5kdc[7261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998121, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:35:21.799630 7282 init.cc:377] Logged in from keytab as kudu/127.7.43.126@KRBTEST.COM (short username kudu)
I20260806 06:35:21.800047 7282 webserver.cc:460] Webserver started at http://127.7.43.126:45821/ using document root <none> and password file <none>
I20260806 06:35:21.800266 7282 fs_manager.cc:359] Metadata directory not provided
I20260806 06:35:21.800312 7282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:35:21.800434 7282 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:35:21.801384 7282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785997861828271-7341-0/minicluster-data/master-0/data/instance:
uuid: "7a6758c9f13345218a092df60c25f63d"
format_stamp: "Formatted at 2026-08-06 06:35:21 on dist-test-slave-6zbq"
server_key: "808fcddb260e2dfd75b5bd34f4fc671f"
server_key_iv: "efb00d16ebed2c77636cb9e69ac5cba2"
server_key_version: "encryptionkey@0"
I20260806 06:35:21.801697 7282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785997861828271-7341-0/minicluster-data/master-0/wal/instance:
uuid: "7a6758c9f13345218a092df60c25f63d"
format_stamp: "Formatted at 2026-08-06 06:35:21 on dist-test-slave-6zbq"
server_key: "808fcddb260e2dfd75b5bd34f4fc671f"
server_key_iv: "efb00d16ebed2c77636cb9e69ac5cba2"
server_key_version: "encryptionkey@0"
I20260806 06:35:21.802942 7282 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:35:21.803572 7306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:21.803719 7282 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:21.803799 7282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "7a6758c9f13345218a092df60c25f63d"
format_stamp: "Formatted at 2026-08-06 06:35:21 on dist-test-slave-6zbq"
server_key: "808fcddb260e2dfd75b5bd34f4fc671f"
server_key_iv: "efb00d16ebed2c77636cb9e69ac5cba2"
server_key_version: "encryptionkey@0"
I20260806 06:35:21.803857 7282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:35:21.824342 7282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:35:21.825867 7282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:35:21.825994 7282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:35:21.829675 7282 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.126:36715
I20260806 06:35:21.829713 7384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.126:36715 every 8 connection(s)
I20260806 06:35:21.830003 7282 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:35:21.830806 7385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:35:21.832834 7385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a6758c9f13345218a092df60c25f63d: Bootstrap starting.
I20260806 06:35:21.833331 7385 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7a6758c9f13345218a092df60c25f63d: Neither blocks nor log segments found. Creating new log.
I20260806 06:35:21.833556 7385 log.cc:824] T 00000000000000000000000000000000 P 7a6758c9f13345218a092df60c25f63d: Log is configured to *not* fsync() on all Append() calls
I20260806 06:35:21.834124 7385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a6758c9f13345218a092df60c25f63d: No bootstrap required, opened a new log
I20260806 06:35:21.835664 7385 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a6758c9f13345218a092df60c25f63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a6758c9f13345218a092df60c25f63d" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36715 } }
I20260806 06:35:21.835798 7385 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7a6758c9f13345218a092df60c25f63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:35:21.835819 7385 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a6758c9f13345218a092df60c25f63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a6758c9f13345218a092df60c25f63d, State: Initialized, Role: FOLLOWER
I20260806 06:35:21.835914 7385 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a6758c9f13345218a092df60c25f63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a6758c9f13345218a092df60c25f63d" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36715 } }
I20260806 06:35:21.835963 7385 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7a6758c9f13345218a092df60c25f63d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:35:21.835983 7385 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a6758c9f13345218a092df60c25f63d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:35:21.836016 7385 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a6758c9f13345218a092df60c25f63d [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:35:21.836746 7385 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a6758c9f13345218a092df60c25f63d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a6758c9f13345218a092df60c25f63d" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36715 } }
I20260806 06:35:21.836833 7385 leader_election.cc:302] T 00000000000000000000000000000000 P 7a6758c9f13345218a092df60c25f63d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7a6758c9f13345218a092df60c25f63d; no voters:
I20260806 06:35:21.836943 7385 leader_election.cc:288] T 00000000000000000000000000000000 P 7a6758c9f13345218a092df60c25f63d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:35:21.837005 7389 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7a6758c9f13345218a092df60c25f63d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:35:21.837165 7389 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7a6758c9f13345218a092df60c25f63d [term 1 LEADER]: Becoming Leader. State: Replica: 7a6758c9f13345218a092df60c25f63d, State: Running, Role: LEADER
I20260806 06:35:21.837184 7385 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a6758c9f13345218a092df60c25f63d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:35:21.837252 7389 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a6758c9f13345218a092df60c25f63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a6758c9f13345218a092df60c25f63d" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36715 } }
I20260806 06:35:21.837577 7390 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a6758c9f13345218a092df60c25f63d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7a6758c9f13345218a092df60c25f63d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a6758c9f13345218a092df60c25f63d" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36715 } } }
I20260806 06:35:21.837603 7395 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a6758c9f13345218a092df60c25f63d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a6758c9f13345218a092df60c25f63d. Latest consensus state: current_term: 1 leader_uuid: "7a6758c9f13345218a092df60c25f63d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a6758c9f13345218a092df60c25f63d" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 36715 } } }
I20260806 06:35:21.837702 7395 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a6758c9f13345218a092df60c25f63d [sys.catalog]: This master's current role is: LEADER
I20260806 06:35:21.837909 7390 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a6758c9f13345218a092df60c25f63d [sys.catalog]: This master's current role is: LEADER
I20260806 06:35:21.837911 7403 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:35:21.838511 7403 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:35:21.838927 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 7282
I20260806 06:35:21.839039 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785997861828271-7341-0/minicluster-data/master-0/wal/instance
I20260806 06:35:21.839228 7341 external_mini_cluster.cc:1428] Setting key aaa5e7f10c2407d75f9f971eded64d35
I20260806 06:35:21.840128 7403 catalog_manager.cc:1347] Generated new cluster ID: 03b25e54334848cda2cf8bd23de88ca2
I20260806 06:35:21.840173 7403 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:35:21.845252 7419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:21.839834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48349 (local address 127.7.43.126:36715)
0806 06:35:21.840190 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:21.840198 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:21.840240 (+ 42us) server_negotiation.cc:407] Connection header received
0806 06:35:21.840346 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:21.840352 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:21.840402 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:21.840539 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:21.840750 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:21.842124 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:21.843162 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:21.843768 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:21.843876 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:21.844473 (+ 597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:21.844477 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:21.844477 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:35:21.844686 (+ 209us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:21.844727 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:35:21.844742 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:21.844807 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:21.844867 (+ 60us) server_negotiation.cc:299] Negotiation successful
0806 06:35:21.844935 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":72,"threads_started":1}
I20260806 06:35:21.846150 7341 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:35:21.868369 7403 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:35:21.868940 7403 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:35:21.869997 7419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:21.866615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48161 (local address 127.7.43.126:36715)
0806 06:35:21.866658 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:21.866660 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:21.866749 (+ 89us) server_negotiation.cc:407] Connection header received
0806 06:35:21.866807 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:21.866808 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:21.866818 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:21.866880 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:21.867027 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:21.867735 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:21.868749 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:21.869045 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:21.869115 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:21.869765 (+ 650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:21.869767 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:21.869767 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:35:21.869773 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:21.869787 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:35:21.869791 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:21.869837 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:21.869887 (+ 50us) server_negotiation.cc:299] Negotiation successful
0806 06:35:21.869924 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 06:35:21.884133 7403 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7a6758c9f13345218a092df60c25f63d: Generated new TSK 0
I20260806 06:35:21.884291 7403 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:35:21.886014 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 7282
2026-08-06T06:35:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1392 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785997861828271-7341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:35:21 dist-test-slave-6zbq krb5kdc[7429](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:35:21 dist-test-slave-6zbq krb5kdc[7429](info): set up 2 sockets
Aug 06 06:35:21 dist-test-slave-6zbq krb5kdc[7429](info): commencing operation
krb5kdc: starting...
W20260806 06:35:22.988422 7341 mini_kdc.cc:121] Time spent starting KDC: real 1.082s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Aug 06 06:35:23 dist-test-slave-6zbq krb5kdc[7429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998123, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:35:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:35:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.43.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
I20260806 06:35:23.094754 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:42643
--webserver_interface=127.7.43.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:46851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.43.126:42643
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:35:23.172621 7445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:35:23.172798 7445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:35:23.172828 7445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:35:23.173913 7445 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:35:23.173951 7445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:35:23.173970 7445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:35:23.173987 7445 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:35:23.174002 7445 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:35:23.175455 7445 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:46851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785997861828271-7341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.43.126:42643
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:42643
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.43.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7445
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:35:23.175678 7445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:35:23.175873 7445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:35:23.180239 7456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:23.180383 7452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:23.180330 7450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:23.180502 7445 server_base.cc:1034] running on GCE node
I20260806 06:35:23.180732 7445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:35:23.180958 7445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:35:23.182092 7445 hybrid_clock.cc:648] HybridClock initialized: now 1785998123182077 us; error 32 us; skew 500 ppm
Aug 06 06:35:23 dist-test-slave-6zbq krb5kdc[7429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998123, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:35:23.184299 7445 init.cc:377] Logged in from keytab as kudu/127.7.43.126@KRBTEST.COM (short username kudu)
I20260806 06:35:23.184696 7445 webserver.cc:460] Webserver started at http://127.7.43.126:41067/ using document root <none> and password file <none>
I20260806 06:35:23.184912 7445 fs_manager.cc:359] Metadata directory not provided
I20260806 06:35:23.184957 7445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:35:23.185078 7445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:35:23.186029 7445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785997861828271-7341-0/minicluster-data/master-0/data/instance:
uuid: "a1078588d9154de2bc5c12643eb35bd7"
format_stamp: "Formatted at 2026-08-06 06:35:23 on dist-test-slave-6zbq"
server_key: "7a3be9b56b11d8875e881a94bd092549"
server_key_iv: "f51d43fabc87b9195dfae84248f26073"
server_key_version: "encryptionkey@0"
I20260806 06:35:23.186367 7445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785997861828271-7341-0/minicluster-data/master-0/wal/instance:
uuid: "a1078588d9154de2bc5c12643eb35bd7"
format_stamp: "Formatted at 2026-08-06 06:35:23 on dist-test-slave-6zbq"
server_key: "7a3be9b56b11d8875e881a94bd092549"
server_key_iv: "f51d43fabc87b9195dfae84248f26073"
server_key_version: "encryptionkey@0"
I20260806 06:35:23.187531 7445 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:35:23.188185 7464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:23.188338 7445 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:35:23.188421 7445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "a1078588d9154de2bc5c12643eb35bd7"
format_stamp: "Formatted at 2026-08-06 06:35:23 on dist-test-slave-6zbq"
server_key: "7a3be9b56b11d8875e881a94bd092549"
server_key_iv: "f51d43fabc87b9195dfae84248f26073"
server_key_version: "encryptionkey@0"
I20260806 06:35:23.188489 7445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:35:23.191577 7445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:35:23.193127 7445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:35:23.193249 7445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:35:23.196799 7445 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.126:42643
I20260806 06:35:23.196827 7543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.126:42643 every 8 connection(s)
I20260806 06:35:23.197091 7445 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:35:23.197911 7544 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:35:23.198275 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 7445
I20260806 06:35:23.198410 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785997861828271-7341-0/minicluster-data/master-0/wal/instance
I20260806 06:35:23.198568 7341 external_mini_cluster.cc:1428] Setting key 5011c39f413bf2ad74a230be97230f63
I20260806 06:35:23.200506 7544 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1078588d9154de2bc5c12643eb35bd7: Bootstrap starting.
I20260806 06:35:23.200709 7549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:23.199013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55269 (local address 127.7.43.126:42643)
0806 06:35:23.199181 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:23.199190 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:23.199222 (+ 32us) server_negotiation.cc:407] Connection header received
0806 06:35:23.199676 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:23.199684 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:23.199716 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:23.199861 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:23.200355 (+ 494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:23.200359 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:23.200359 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:35:23.200363 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:23.200401 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:35:23.200415 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:23.200486 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:23.200517 (+ 31us) server_negotiation.cc:299] Negotiation successful
0806 06:35:23.200553 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":69,"threads_started":1}
I20260806 06:35:23.201133 7544 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a1078588d9154de2bc5c12643eb35bd7: Neither blocks nor log segments found. Creating new log.
I20260806 06:35:23.201421 7544 log.cc:824] T 00000000000000000000000000000000 P a1078588d9154de2bc5c12643eb35bd7: Log is configured to *not* fsync() on all Append() calls
I20260806 06:35:23.201658 7341 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:35:23.202147 7544 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1078588d9154de2bc5c12643eb35bd7: No bootstrap required, opened a new log
I20260806 06:35:23.204023 7549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:23.203158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35263 (local address 127.7.43.126:42643)
0806 06:35:23.203198 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:23.203200 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:23.203238 (+ 38us) server_negotiation.cc:407] Connection header received
0806 06:35:23.203312 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:23.203313 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:23.203316 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:23.203367 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:23.203837 (+ 470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:23.203838 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:23.203838 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:35:23.203842 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:23.203855 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:35:23.203859 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:23.203904 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:23.203953 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 06:35:23.203965 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 06:35:23.204149 7544 raft_consensus.cc:348] T 00000000000000000000000000000000 P a1078588d9154de2bc5c12643eb35bd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1078588d9154de2bc5c12643eb35bd7" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 42643 } }
I20260806 06:35:23.204304 7544 raft_consensus.cc:374] T 00000000000000000000000000000000 P a1078588d9154de2bc5c12643eb35bd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:35:23.204351 7544 raft_consensus.cc:729] T 00000000000000000000000000000000 P a1078588d9154de2bc5c12643eb35bd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1078588d9154de2bc5c12643eb35bd7, State: Initialized, Role: FOLLOWER
I20260806 06:35:23.204464 7544 consensus_queue.cc:260] T 00000000000000000000000000000000 P a1078588d9154de2bc5c12643eb35bd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1078588d9154de2bc5c12643eb35bd7" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 42643 } }
I20260806 06:35:23.204542 7544 raft_consensus.cc:388] T 00000000000000000000000000000000 P a1078588d9154de2bc5c12643eb35bd7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:35:23.204573 7544 raft_consensus.cc:482] T 00000000000000000000000000000000 P a1078588d9154de2bc5c12643eb35bd7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:35:23.204619 7544 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a1078588d9154de2bc5c12643eb35bd7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:35:23.205096 7544 raft_consensus.cc:504] T 00000000000000000000000000000000 P a1078588d9154de2bc5c12643eb35bd7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1078588d9154de2bc5c12643eb35bd7" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 42643 } }
I20260806 06:35:23.205188 7544 leader_election.cc:302] T 00000000000000000000000000000000 P a1078588d9154de2bc5c12643eb35bd7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a1078588d9154de2bc5c12643eb35bd7; no voters:
I20260806 06:35:23.205327 7544 leader_election.cc:288] T 00000000000000000000000000000000 P a1078588d9154de2bc5c12643eb35bd7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:35:23.205360 7558 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a1078588d9154de2bc5c12643eb35bd7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:35:23.205504 7558 raft_consensus.cc:686] T 00000000000000000000000000000000 P a1078588d9154de2bc5c12643eb35bd7 [term 1 LEADER]: Becoming Leader. State: Replica: a1078588d9154de2bc5c12643eb35bd7, State: Running, Role: LEADER
I20260806 06:35:23.205535 7544 sys_catalog.cc:564] T 00000000000000000000000000000000 P a1078588d9154de2bc5c12643eb35bd7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:35:23.205627 7558 consensus_queue.cc:237] T 00000000000000000000000000000000 P a1078588d9154de2bc5c12643eb35bd7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1078588d9154de2bc5c12643eb35bd7" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 42643 } }
I20260806 06:35:23.206009 7560 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1078588d9154de2bc5c12643eb35bd7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a1078588d9154de2bc5c12643eb35bd7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1078588d9154de2bc5c12643eb35bd7" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 42643 } } }
I20260806 06:35:23.206159 7560 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1078588d9154de2bc5c12643eb35bd7 [sys.catalog]: This master's current role is: LEADER
I20260806 06:35:23.206022 7561 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1078588d9154de2bc5c12643eb35bd7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a1078588d9154de2bc5c12643eb35bd7. Latest consensus state: current_term: 1 leader_uuid: "a1078588d9154de2bc5c12643eb35bd7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1078588d9154de2bc5c12643eb35bd7" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 42643 } } }
I20260806 06:35:23.206409 7561 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1078588d9154de2bc5c12643eb35bd7 [sys.catalog]: This master's current role is: LEADER
I20260806 06:35:23.206552 7574 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:35:23.206987 7574 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:35:23.208473 7574 catalog_manager.cc:1347] Generated new cluster ID: 525611258591407d9fb52090f2ce1eff
I20260806 06:35:23.208519 7574 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:35:23.220507 7574 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:35:23.220651 7574 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:35:23.228134 7574 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a1078588d9154de2bc5c12643eb35bd7: Generated new TSK 0
I20260806 06:35:23.228302 7574 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:35:23.233911 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 7445
2026-08-06T06:35:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1360 ms)
[----------] 18 tests from AuthTokenIssuingTest (24701 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:35:23 dist-test-slave-6zbq krb5kdc[7590](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:35:23 dist-test-slave-6zbq krb5kdc[7590](info): set up 2 sockets
Aug 06 06:35:23 dist-test-slave-6zbq krb5kdc[7590](info): commencing operation
krb5kdc: starting...
W20260806 06:35:24.353430 7341 mini_kdc.cc:121] Time spent starting KDC: real 1.086s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Aug 06 06:35:24 dist-test-slave-6zbq krb5kdc[7590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998124, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:35:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:35:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.43.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
I20260806 06:35:24.458473 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:43259
--webserver_interface=127.7.43.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:34423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.43.126:43259,127.7.43.125:37521,127.7.43.124:46385
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:35:24.535341 7606 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 06:35:24.535522 7606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:35:24.535538 7606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:35:24.535555 7606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:35:24.536602 7606 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:35:24.536631 7606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:35:24.536644 7606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:35:24.536655 7606 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:35:24.536671 7606 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:35:24.538029 7606 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:34423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.43.126:43259,127.7.43.125:37521,127.7.43.124:46385
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:43259
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.43.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7606
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:35:24.538198 7606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:35:24.538384 7606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:35:24.542798 7613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:24.542904 7606 server_base.cc:1034] running on GCE node
W20260806 06:35:24.542807 7618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:24.542836 7614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:24.543182 7606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:35:24.543457 7606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:35:24.544593 7606 hybrid_clock.cc:648] HybridClock initialized: now 1785998124544575 us; error 30 us; skew 500 ppm
Aug 06 06:35:24 dist-test-slave-6zbq krb5kdc[7590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998124, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:35:24.546854 7606 init.cc:377] Logged in from keytab as kudu/127.7.43.126@KRBTEST.COM (short username kudu)
I20260806 06:35:24.547291 7606 webserver.cc:460] Webserver started at http://127.7.43.126:36371/ using document root <none> and password file <none>
I20260806 06:35:24.547483 7606 fs_manager.cc:359] Metadata directory not provided
I20260806 06:35:24.547528 7606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:35:24.547652 7606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:35:24.548601 7606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/minicluster-data/master-0/data/instance:
uuid: "3024873d038b4cad85afce8303eff7b4"
format_stamp: "Formatted at 2026-08-06 06:35:24 on dist-test-slave-6zbq"
server_key: "86c3317f163a1430f38bf86ef737c1ff"
server_key_iv: "9aadf269efb71eb45a9c769cc3910114"
server_key_version: "encryptionkey@0"
I20260806 06:35:24.548923 7606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/minicluster-data/master-0/wal/instance:
uuid: "3024873d038b4cad85afce8303eff7b4"
format_stamp: "Formatted at 2026-08-06 06:35:24 on dist-test-slave-6zbq"
server_key: "86c3317f163a1430f38bf86ef737c1ff"
server_key_iv: "9aadf269efb71eb45a9c769cc3910114"
server_key_version: "encryptionkey@0"
I20260806 06:35:24.550311 7606 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:35:24.551082 7627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:24.551226 7606 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:35:24.551324 7606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "3024873d038b4cad85afce8303eff7b4"
format_stamp: "Formatted at 2026-08-06 06:35:24 on dist-test-slave-6zbq"
server_key: "86c3317f163a1430f38bf86ef737c1ff"
server_key_iv: "9aadf269efb71eb45a9c769cc3910114"
server_key_version: "encryptionkey@0"
I20260806 06:35:24.551393 7606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:35:24.566124 7606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:35:24.567721 7606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:35:24.567842 7606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:35:24.571425 7606 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.126:43259
I20260806 06:35:24.571455 7709 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.126:43259 every 8 connection(s)
I20260806 06:35:24.571733 7606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:35:24.572069 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 7606
I20260806 06:35:24.572180 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/minicluster-data/master-0/wal/instance
I20260806 06:35:24.572389 7341 external_mini_cluster.cc:1428] Setting key ace91b553c103e1ad9a1d244dd1debd5
I20260806 06:35:24.572796 7711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:35:24.574548 7711 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.43.126" port: 43259 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.7.43.125@KRBTEST.COM; defaulting to no policy
Aug 06 06:35:24 dist-test-slave-6zbq krb5kdc[7590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998124, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.126@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.125@KRBTEST.COM" created.
I20260806 06:35:24.583240 7714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:24.575182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:43259 (local address 127.0.0.1:59112)
0806 06:35:24.575327 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:24.575356 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:24.575806 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:24.576050 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:24.576053 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:24.576299 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:35:24.576438 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:24.576442 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:24.577394 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:24.577395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:24.578025 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:24.578026 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:24.578526 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:24.578527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:24.578591 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:24.579026 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:35:24.579034 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:35:24.580292 (+ 1258us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:35:24.581930 (+ 1638us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:24.581936 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:24.581938 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:24.582148 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:24.582367 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:24.582368 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:24.582368 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:24.582452 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:24.582748 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:35:24.582752 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:35:24.582811 (+ 59us) client_negotiation.cc:769] Sending connection context
0806 06:35:24.582890 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 06:35:24.583100 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":84,"thread_start_us":74,"threads_started":1}
I20260806 06:35:24.583247 7715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:24.575355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59112 (local address 127.7.43.126:43259)
0806 06:35:24.575475 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:24.575479 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:24.575509 (+ 30us) server_negotiation.cc:407] Connection header received
0806 06:35:24.575943 (+ 434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:24.575945 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:24.575962 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:24.576039 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:24.576526 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:24.577310 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:24.578101 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:24.578428 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:24.578516 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:24.580380 (+ 1864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:24.580387 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:24.580388 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:24.580394 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:24.581855 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:24.582217 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:24.582220 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:24.582220 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:24.582296 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:24.582504 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:24.582504 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:24.582505 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:24.582688 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:24.582755 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:24.582930 (+ 175us) server_negotiation.cc:299] Negotiation successful
0806 06:35:24.583029 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":56,"threads_started":1}
I20260806 06:35:24.583841 7711 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.43.125" port: 37521 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:35:24.584139 7714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:24.584023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.125:37521 (local address 127.0.0.1:40588)
0806 06:35:24.584068 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:24.584115 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.125:37521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260806 06:35:24.584286 7630 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.43.125:37521: connect: Connection refused (error 111)
I20260806 06:35:24.584671 7714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:24.584576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.125:37521 (local address 127.0.0.1:40600)
0806 06:35:24.584621 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:24.584649 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.125:37521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 06:35:24.584769 7711 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.43.125:37521: Network error: Client connection negotiation failed: client connection to 127.7.43.125:37521: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.125.keytab.
Entry for principal kudu/127.7.43.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.125.keytab.
I20260806 06:35:24.594583 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.125:37521
--webserver_interface=127.7.43.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.125.keytab
--principal=kudu/127.7.43.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:34423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.43.126:43259,127.7.43.125:37521,127.7.43.124:46385
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:35:24.633960 7711 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.43.125" port: 37521 } attempt: 1
I20260806 06:35:24.634385 7714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:24.634241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.125:37521 (local address 127.0.0.1:40614)
0806 06:35:24.634296 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:24.634328 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.125:37521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 06:35:24.634673 7714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:24.634606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.125:37521 (local address 127.0.0.1:40626)
0806 06:35:24.634636 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:24.634658 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.125:37521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260806 06:35:24.634768 7711 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.43.125:37521: Network error: Client connection negotiation failed: client connection to 127.7.43.125:37521: connect: Connection refused (error 111)
W20260806 06:35:24.674911 7720 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 06:35:24.675086 7720 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:35:24.675108 7720 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:35:24.675133 7720 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:35:24.676208 7720 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:35:24.676254 7720 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:35:24.676273 7720 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:35:24.676290 7720 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:35:24.676306 7720 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:35:24.677727 7720 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:34423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.43.126:43259,127.7.43.125:37521,127.7.43.124:46385
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.125.keytab
--principal=kudu/127.7.43.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.125:37521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.43.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7720
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:35:24.677943 7720 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:35:24.678131 7720 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:35:24.682504 7720 server_base.cc:1034] running on GCE node
W20260806 06:35:24.682451 7739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:24.682451 7737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:24.682471 7735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:24.682885 7720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:35:24.683125 7720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:35:24.684259 7720 hybrid_clock.cc:648] HybridClock initialized: now 1785998124684242 us; error 29 us; skew 500 ppm
Aug 06 06:35:24 dist-test-slave-6zbq krb5kdc[7590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998124, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:35:24.686455 7720 init.cc:377] Logged in from keytab as kudu/127.7.43.125@KRBTEST.COM (short username kudu)
I20260806 06:35:24.686851 7720 webserver.cc:460] Webserver started at http://127.7.43.125:40759/ using document root <none> and password file <none>
I20260806 06:35:24.687052 7720 fs_manager.cc:359] Metadata directory not provided
I20260806 06:35:24.687098 7720 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:35:24.687214 7720 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:35:24.688131 7720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/minicluster-data/master-1/data/instance:
uuid: "a8b6a480861246abb83589b127e50c0a"
format_stamp: "Formatted at 2026-08-06 06:35:24 on dist-test-slave-6zbq"
server_key: "45c8361cfb846a92471859fd0c06d4f5"
server_key_iv: "6baee68aff3b0b9e158fc557b0e5da04"
server_key_version: "encryptionkey@0"
I20260806 06:35:24.688441 7720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/minicluster-data/master-1/wal/instance:
uuid: "a8b6a480861246abb83589b127e50c0a"
format_stamp: "Formatted at 2026-08-06 06:35:24 on dist-test-slave-6zbq"
server_key: "45c8361cfb846a92471859fd0c06d4f5"
server_key_iv: "6baee68aff3b0b9e158fc557b0e5da04"
server_key_version: "encryptionkey@0"
I20260806 06:35:24.689556 7720 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:35:24.690172 7748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:24.690322 7720 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:24.690425 7720 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/minicluster-data/master-1/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/minicluster-data/master-1/wal
uuid: "a8b6a480861246abb83589b127e50c0a"
format_stamp: "Formatted at 2026-08-06 06:35:24 on dist-test-slave-6zbq"
server_key: "45c8361cfb846a92471859fd0c06d4f5"
server_key_iv: "6baee68aff3b0b9e158fc557b0e5da04"
server_key_version: "encryptionkey@0"
I20260806 06:35:24.690487 7720 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:35:24.702891 7711 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.43.125" port: 37521 } attempt: 2
I20260806 06:35:24.703202 7714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:24.703100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.125:37521 (local address 127.0.0.1:40640)
0806 06:35:24.703149 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:24.703175 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.125:37521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260806 06:35:24.703510 7714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:24.703429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.125:37521 (local address 127.0.0.1:40644)
0806 06:35:24.703470 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:24.703492 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.125:37521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260806 06:35:24.703585 7711 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.43.125:37521: Network error: Client connection negotiation failed: client connection to 127.7.43.125:37521: connect: Connection refused (error 111)
I20260806 06:35:24.710712 7720 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:35:24.712177 7720 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:35:24.712308 7720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:35:24.716012 7720 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.125:37521
I20260806 06:35:24.716040 7814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.125:37521 every 8 connection(s)
I20260806 06:35:24.716333 7720 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/minicluster-data/master-1/data/info.pb
I20260806 06:35:24.717180 7816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:35:24.718497 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 7720
I20260806 06:35:24.718607 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/minicluster-data/master-1/wal/instance
I20260806 06:35:24.718779 7341 external_mini_cluster.cc:1428] Setting key 6fe21c36d1ae40b86d3273d7262cfedf
I20260806 06:35:24.718729 7816 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.43.126" port: 43259 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:35:24 dist-test-slave-6zbq krb5kdc[7590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998124, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.125@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.43.124@KRBTEST.COM; defaulting to no policy
I20260806 06:35:24.727198 7715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:24.719419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59116 (local address 127.7.43.126:43259)
0806 06:35:24.719476 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:24.719478 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:24.719661 (+ 183us) server_negotiation.cc:407] Connection header received
0806 06:35:24.720205 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:24.720206 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:24.720217 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:24.720269 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:24.720945 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:24.721647 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:24.722589 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:24.722792 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:24.722849 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:24.724704 (+ 1855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:24.724705 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:24.724706 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:24.724713 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:24.725925 (+ 1212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:24.726288 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:24.726289 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:24.726289 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:24.726370 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:24.726627 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:24.726628 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:24.726628 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:24.726736 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:24.726790 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:24.727034 (+ 244us) server_negotiation.cc:299] Negotiation successful
0806 06:35:24.727116 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 06:35:24.727378 7818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:24.719382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:43259 (local address 127.0.0.1:59116)
0806 06:35:24.719534 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:24.719578 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:24.720112 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:24.720339 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:24.720342 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:24.720667 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:35:24.720854 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:24.720859 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:24.721715 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:24.721715 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:24.722511 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:24.722514 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:24.722883 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:24.722884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:24.722993 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:24.723404 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:35:24.723412 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:35:24.724620 (+ 1208us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:35:24.725997 (+ 1377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:24.726009 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:24.726011 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:24.726211 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:24.726435 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:24.726436 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:24.726436 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:24.726562 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:24.726794 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:35:24.726801 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:35:24.726922 (+ 121us) client_negotiation.cc:769] Sending connection context
0806 06:35:24.727007 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 06:35:24.727183 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":74,"threads_started":1}
I20260806 06:35:24.727851 7816 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.43.125" port: 37521 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.124@KRBTEST.COM" created.
Aug 06 06:35:24 dist-test-slave-6zbq krb5kdc[7590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998124, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.125@KRBTEST.COM for kudu/127.7.43.125@KRBTEST.COM
I20260806 06:35:24.734869 7818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:24.728054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.125:37521 (local address 127.0.0.1:40650)
0806 06:35:24.728092 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:24.728136 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:24.728175 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:24.728410 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:24.728411 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:24.728512 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:35:24.728558 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:24.728559 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:24.729496 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:24.729498 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:24.730191 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:24.730194 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:24.730842 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:24.730843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:24.730898 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:24.731219 (+ 321us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:35:24.731223 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:35:24.732336 (+ 1113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:35:24.733844 (+ 1508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:24.733845 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:24.733845 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:24.734050 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:24.734222 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:24.734222 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:24.734223 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:24.734287 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:24.734544 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:35:24.734545 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:35:24.734590 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 06:35:24.734670 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 06:35:24.734760 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260806 06:35:24.734889 7820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:24.728109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40650 (local address 127.7.43.125:37521)
0806 06:35:24.728297 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:24.728300 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:24.728317 (+ 17us) server_negotiation.cc:407] Connection header received
0806 06:35:24.728335 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:24.728337 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:24.728347 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:24.728407 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:24.728611 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:24.729412 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:24.730293 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:24.730753 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:24.730827 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:24.732416 (+ 1589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:24.732423 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:24.732424 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:24.732430 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:24.733759 (+ 1329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:24.734119 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:24.734121 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:24.734121 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:24.734167 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:24.734359 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:24.734359 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:24.734360 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:24.734482 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:24.734541 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:24.734693 (+ 152us) server_negotiation.cc:299] Negotiation successful
0806 06:35:24.734806 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":53,"server-negotiator.queue_time_us":162,"thread_start_us":78,"threads_started":1}
I20260806 06:35:24.735453 7816 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.43.124" port: 46385 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:35:24.735726 7818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:24.735621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.124:46385 (local address 127.0.0.1:53824)
0806 06:35:24.735653 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:24.735703 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.124:46385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260806 06:35:24.735860 7749 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.43.124:46385: connect: Connection refused (error 111)
I20260806 06:35:24.736234 7818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:24.736143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.124:46385 (local address 127.0.0.1:53836)
0806 06:35:24.736191 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:24.736217 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.124:46385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 06:35:24.736323 7816 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.43.124:46385: Network error: Client connection negotiation failed: client connection to 127.7.43.124:46385: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.124.keytab.
Entry for principal kudu/127.7.43.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.124.keytab.
I20260806 06:35:24.742110 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.124:46385
--webserver_interface=127.7.43.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.124.keytab
--principal=kudu/127.7.43.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:34423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.43.126:43259,127.7.43.125:37521,127.7.43.124:46385
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:35:24.785532 7816 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.43.124" port: 46385 } attempt: 1
I20260806 06:35:24.785969 7818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:24.785837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.124:46385 (local address 127.0.0.1:53848)
0806 06:35:24.785897 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:24.785937 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.124:46385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 06:35:24.786284 7818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:24.786187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.124:46385 (local address 127.0.0.1:53860)
0806 06:35:24.786227 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:24.786265 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.124:46385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260806 06:35:24.786387 7816 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.43.124:46385: Network error: Client connection negotiation failed: client connection to 127.7.43.124:46385: connect: Connection refused (error 111)
I20260806 06:35:24.794700 7711 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.43.125" port: 37521 } attempt: 3
Aug 06 06:35:24 dist-test-slave-6zbq krb5kdc[7590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998124, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.126@KRBTEST.COM for kudu/127.7.43.125@KRBTEST.COM
I20260806 06:35:24.801352 7714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:24.794955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.125:37521 (local address 127.0.0.1:40658)
0806 06:35:24.795003 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:24.795013 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:24.795050 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:24.795183 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:24.795184 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:24.795340 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:35:24.795409 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:24.795412 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:24.796352 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:24.796352 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:24.796887 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:24.796888 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:24.797359 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:24.797360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:24.797409 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:24.797836 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:35:24.797840 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:35:24.798978 (+ 1138us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:35:24.800560 (+ 1582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:24.800561 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:24.800561 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:24.800714 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:24.800879 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:24.800879 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:24.800880 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:24.800922 (+ 42us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:24.801138 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:35:24.801139 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:35:24.801173 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 06:35:24.801217 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 06:35:24.801288 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 06:35:24.801410 7820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:24.795035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40658 (local address 127.7.43.125:37521)
0806 06:35:24.795086 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:24.795088 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:24.795095 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:35:24.795120 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:24.795121 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:24.795130 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:24.795173 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:24.795495 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:24.796262 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:24.796956 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:24.797294 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:24.797366 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:24.799058 (+ 1692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:24.799059 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:24.799060 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:24.799067 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:24.800474 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:24.800769 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:24.800770 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:24.800770 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:24.800827 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:24.800955 (+ 128us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:24.800955 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:24.800956 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:24.801074 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:24.801139 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:24.801249 (+ 110us) server_negotiation.cc:299] Negotiation successful
0806 06:35:24.801323 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 06:35:24.801690 7711 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.43.124" port: 46385 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:35:24.802100 7714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:24.801956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.124:46385 (local address 127.0.0.1:53870)
0806 06:35:24.802019 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:24.802054 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.124:46385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 06:35:24.802450 7714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:24.802367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.124:46385 (local address 127.0.0.1:53874)
0806 06:35:24.802399 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:24.802427 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.124:46385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260806 06:35:24.802537 7711 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.43.124:46385: Network error: Client connection negotiation failed: client connection to 127.7.43.124:46385: connect: Connection refused (error 111)
W20260806 06:35:24.823375 7823 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 06:35:24.823551 7823 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:35:24.823575 7823 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:35:24.823607 7823 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:35:24.824672 7823 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:35:24.824710 7823 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:35:24.824729 7823 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:35:24.824754 7823 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:35:24.824777 7823 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:35:24.826200 7823 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:34423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.43.126:43259,127.7.43.125:37521,127.7.43.124:46385
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.124.keytab
--principal=kudu/127.7.43.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.124:46385
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.43.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7823
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:35:24.826442 7823 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:35:24.826634 7823 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:35:24.830842 7832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:24.830848 7834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:24.830857 7831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:24.830926 7823 server_base.cc:1034] running on GCE node
I20260806 06:35:24.831240 7823 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:35:24.831462 7823 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:35:24.832593 7823 hybrid_clock.cc:648] HybridClock initialized: now 1785998124832578 us; error 29 us; skew 500 ppm
I20260806 06:35:24.833647 7711 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.43.124" port: 46385 } attempt: 1
I20260806 06:35:24.833962 7714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:24.833872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.124:46385 (local address 127.0.0.1:53890)
0806 06:35:24.833916 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:24.833943 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.124:46385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 06:35:24.834232 7714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:24.834152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.124:46385 (local address 127.0.0.1:53894)
0806 06:35:24.834182 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:24.834210 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.124:46385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260806 06:35:24.834364 7711 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.43.124:46385: Network error: Client connection negotiation failed: client connection to 127.7.43.124:46385: connect: Connection refused (error 111)
Aug 06 06:35:24 dist-test-slave-6zbq krb5kdc[7590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998124, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:35:24.834903 7823 init.cc:377] Logged in from keytab as kudu/127.7.43.124@KRBTEST.COM (short username kudu)
I20260806 06:35:24.835281 7823 webserver.cc:460] Webserver started at http://127.7.43.124:39883/ using document root <none> and password file <none>
I20260806 06:35:24.835489 7823 fs_manager.cc:359] Metadata directory not provided
I20260806 06:35:24.835533 7823 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:35:24.835654 7823 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:35:24.836588 7823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/minicluster-data/master-2/data/instance:
uuid: "4022cbe9ccd743ea9384daf248e44eda"
format_stamp: "Formatted at 2026-08-06 06:35:24 on dist-test-slave-6zbq"
server_key: "626e4a2c889742dabb1be3997572a6d2"
server_key_iv: "1ae52a09a0ba3b98d300bc7d05864927"
server_key_version: "encryptionkey@0"
I20260806 06:35:24.836892 7823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/minicluster-data/master-2/wal/instance:
uuid: "4022cbe9ccd743ea9384daf248e44eda"
format_stamp: "Formatted at 2026-08-06 06:35:24 on dist-test-slave-6zbq"
server_key: "626e4a2c889742dabb1be3997572a6d2"
server_key_iv: "1ae52a09a0ba3b98d300bc7d05864927"
server_key_version: "encryptionkey@0"
I20260806 06:35:24.838028 7823 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:35:24.838683 7844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:24.838828 7823 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:24.838907 7823 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/minicluster-data/master-2/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/minicluster-data/master-2/wal
uuid: "4022cbe9ccd743ea9384daf248e44eda"
format_stamp: "Formatted at 2026-08-06 06:35:24 on dist-test-slave-6zbq"
server_key: "626e4a2c889742dabb1be3997572a6d2"
server_key_iv: "1ae52a09a0ba3b98d300bc7d05864927"
server_key_version: "encryptionkey@0"
I20260806 06:35:24.838969 7823 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:35:24.854498 7816 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.43.124" port: 46385 } attempt: 2
I20260806 06:35:24.854820 7818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:24.854701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.124:46385 (local address 127.0.0.1:53910)
0806 06:35:24.854756 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:24.854789 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.124:46385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 06:35:24.855083 7818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:24.855020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.124:46385 (local address 127.0.0.1:53922)
0806 06:35:24.855051 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:24.855069 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.124:46385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260806 06:35:24.855178 7816 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.43.124:46385: Network error: Client connection negotiation failed: client connection to 127.7.43.124:46385: connect: Connection refused (error 111)
I20260806 06:35:24.862226 7823 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:35:24.863766 7823 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:35:24.863878 7823 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:35:24.867542 7823 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.124:46385
I20260806 06:35:24.867573 7918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.124:46385 every 8 connection(s)
I20260806 06:35:24.867858 7823 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/minicluster-data/master-2/data/info.pb
I20260806 06:35:24.868716 7919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:35:24.870059 7919 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.43.126" port: 43259 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:35:24 dist-test-slave-6zbq krb5kdc[7590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998124, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.124@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:35:24.876386 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 7823
I20260806 06:35:24.876495 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785997861828271-7341-0/minicluster-data/master-2/wal/instance
I20260806 06:35:24.876673 7341 external_mini_cluster.cc:1428] Setting key 48446006a2bd68f09131c9b35f588cf8
I20260806 06:35:24.877707 7715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:24.870652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59126 (local address 127.7.43.126:43259)
0806 06:35:24.870704 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:24.870706 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:24.870887 (+ 181us) server_negotiation.cc:407] Connection header received
0806 06:35:24.871306 (+ 419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:24.871306 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:24.871314 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:24.871359 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:24.871890 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:24.872328 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:24.873222 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:24.873392 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:24.873431 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:24.875272 (+ 1841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:24.875273 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:24.875273 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:24.875279 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:24.876309 (+ 1030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:24.876765 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:24.876766 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:24.876766 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:24.876826 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:24.877085 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:24.877087 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:24.877087 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:24.877226 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:24.877296 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:24.877583 (+ 287us) server_negotiation.cc:299] Negotiation successful
0806 06:35:24.877648 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 06:35:24.877924 7921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:24.870637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:43259 (local address 127.0.0.1:59126)
0806 06:35:24.870786 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:24.870818 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:24.871237 (+ 419us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:24.871422 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:24.871425 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:24.871653 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:35:24.871826 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:24.871830 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:24.872392 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:24.872394 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:24.873161 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:24.873163 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:24.873452 (+ 289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:24.873453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:24.873545 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:24.873964 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:35:24.873972 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:35:24.875201 (+ 1229us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:35:24.876378 (+ 1177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:24.876385 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:24.876387 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:24.876669 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:24.876890 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:24.876891 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:24.876891 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:24.877019 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:24.877297 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:35:24.877303 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:35:24.877452 (+ 149us) client_negotiation.cc:769] Sending connection context
0806 06:35:24.877560 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 06:35:24.877705 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":71,"threads_started":1}
I20260806 06:35:24.878504 7919 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.43.125" port: 37521 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:35:24 dist-test-slave-6zbq krb5kdc[7590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998124, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.124@KRBTEST.COM for kudu/127.7.43.125@KRBTEST.COM
Aug 06 06:35:24 dist-test-slave-6zbq krb5kdc[7590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998124, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.43.124@KRBTEST.COM
I20260806 06:35:24.884214 7820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:24.878771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40662 (local address 127.7.43.125:37521)
0806 06:35:24.878819 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:24.878821 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:24.878828 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:35:24.878849 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:24.878850 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:24.878859 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:24.878903 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:24.879151 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:24.879581 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:24.880189 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:24.880397 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:24.880447 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:24.881936 (+ 1489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:24.881938 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:24.881938 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:24.881944 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:24.883320 (+ 1376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:24.883622 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:24.883623 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:24.883624 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:24.883677 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:24.883813 (+ 136us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:24.883814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:24.883814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:24.883935 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:24.884002 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:24.884106 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 06:35:24.884160 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:35:24.884245 7921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:24.878709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.125:37521 (local address 127.0.0.1:40662)
0806 06:35:24.878744 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:24.878753 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:24.878787 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:24.878913 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:24.878914 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:24.879032 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:35:24.879073 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:24.879074 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:24.879647 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:24.879648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:24.880122 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:24.880124 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:24.880452 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:24.880453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:24.880510 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:24.880816 (+ 306us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:35:24.880819 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:35:24.881861 (+ 1042us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:35:24.883395 (+ 1534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:24.883396 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:24.883396 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:24.883559 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:24.883735 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:24.883735 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:24.883735 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:24.883777 (+ 42us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:24.883992 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:35:24.883992 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:35:24.884024 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 06:35:24.884083 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 06:35:24.884160 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260806 06:35:24.884549 7919 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.43.124" port: 46385 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:35:24.885243 7925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:24.877018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53934 (local address 127.7.43.124:46385)
0806 06:35:24.877150 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:24.877154 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:24.877265 (+ 111us) server_negotiation.cc:407] Connection header received
0806 06:35:24.877310 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:24.877312 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:24.877325 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:24.877383 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:24.878077 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:24.878912 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:24.879710 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:24.880003 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:24.880059 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:24.882557 (+ 2498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:24.882565 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:24.882565 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:24.882572 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:24.883984 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:24.884357 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:24.884361 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:24.884361 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:24.884417 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:24.884576 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:24.884577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:24.884577 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:24.884734 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:24.884857 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:24.885052 (+ 195us) server_negotiation.cc:299] Negotiation successful
0806 06:35:24.885133 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":69,"threads_started":1}
Aug 06 06:35:24 dist-test-slave-6zbq krb5kdc[7590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998124, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.124@KRBTEST.COM for kudu/127.7.43.124@KRBTEST.COM
Aug 06 06:35:24 dist-test-slave-6zbq krb5kdc[7590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998124, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:35:24.891055 7921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:24.884756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.124:46385 (local address 127.0.0.1:53942)
0806 06:35:24.884809 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:24.884818 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:24.884846 (+ 28us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:24.885014 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:24.885015 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:24.885153 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:35:24.885180 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:24.885182 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:24.885888 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:24.885889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:24.886335 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:24.886361 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:24.886686 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:24.886687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:24.886732 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:24.887047 (+ 315us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:35:24.887052 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:35:24.888071 (+ 1019us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:35:24.890098 (+ 2027us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:24.890099 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:24.890100 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:24.890291 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:24.890494 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:24.890495 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:24.890495 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:24.890557 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:24.890769 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:35:24.890770 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:35:24.890804 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 06:35:24.890868 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 06:35:24.890956 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260806 06:35:24.891127 7927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:24.884804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53942 (local address 127.7.43.124:46385)
0806 06:35:24.884922 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:24.884924 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:24.884930 (+ 6us) server_negotiation.cc:407] Connection header received
0806 06:35:24.884945 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:24.884945 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:24.884952 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:24.885004 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:24.885256 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:24.885828 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:24.886425 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:24.886633 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:24.886680 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:24.888162 (+ 1482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:24.888163 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:24.888163 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:24.888169 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:24.890035 (+ 1866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:24.890374 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:24.890375 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:24.890375 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:24.890434 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:24.890618 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:24.890618 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:24.890619 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:24.890713 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:24.890770 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:24.890898 (+ 128us) server_negotiation.cc:299] Negotiation successful
0806 06:35:24.891003 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":102,"thread_start_us":68,"threads_started":1}
I20260806 06:35:24.892356 7715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:24.885868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59134 (local address 127.7.43.126:43259)
0806 06:35:24.885902 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:24.885904 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:24.885942 (+ 38us) server_negotiation.cc:407] Connection header received
0806 06:35:24.886003 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:24.886004 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:24.886012 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:24.886054 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:24.886490 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:24.886912 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:24.887509 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:24.887698 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:24.887747 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:24.890027 (+ 2280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:24.890028 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:24.890028 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:24.890034 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:24.891446 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:24.891724 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:24.891725 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:24.891725 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:24.891778 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:24.891934 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:24.891935 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:24.891936 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:24.892032 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:24.892166 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:24.892206 (+ 40us) server_negotiation.cc:299] Negotiation successful
0806 06:35:24.892278 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260806 06:35:24.892609 7919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4022cbe9ccd743ea9384daf248e44eda: Bootstrap starting.
I20260806 06:35:24.893288 7919 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4022cbe9ccd743ea9384daf248e44eda: Neither blocks nor log segments found. Creating new log.
I20260806 06:35:24.893620 7919 log.cc:824] T 00000000000000000000000000000000 P 4022cbe9ccd743ea9384daf248e44eda: Log is configured to *not* fsync() on all Append() calls
I20260806 06:35:24.894383 7919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4022cbe9ccd743ea9384daf248e44eda: No bootstrap required, opened a new log
I20260806 06:35:24.896250 7919 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4022cbe9ccd743ea9384daf248e44eda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3024873d038b4cad85afce8303eff7b4" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 43259 } } peers { permanent_uuid: "a8b6a480861246abb83589b127e50c0a" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 37521 } } peers { permanent_uuid: "4022cbe9ccd743ea9384daf248e44eda" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 46385 } }
I20260806 06:35:24.896400 7919 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4022cbe9ccd743ea9384daf248e44eda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4022cbe9ccd743ea9384daf248e44eda, State: Initialized, Role: FOLLOWER
I20260806 06:35:24.896485 7919 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4022cbe9ccd743ea9384daf248e44eda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3024873d038b4cad85afce8303eff7b4" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 43259 } } peers { permanent_uuid: "a8b6a480861246abb83589b127e50c0a" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 37521 } } peers { permanent_uuid: "4022cbe9ccd743ea9384daf248e44eda" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 46385 } }
I20260806 06:35:24.896704 7929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4022cbe9ccd743ea9384daf248e44eda [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3024873d038b4cad85afce8303eff7b4" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 43259 } } peers { permanent_uuid: "a8b6a480861246abb83589b127e50c0a" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 37521 } } peers { permanent_uuid: "4022cbe9ccd743ea9384daf248e44eda" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 46385 } } }
I20260806 06:35:24.896775 7919 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4022cbe9ccd743ea9384daf248e44eda [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:35:24.896823 7929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4022cbe9ccd743ea9384daf248e44eda [sys.catalog]: This master's current role is: FOLLOWER
Aug 06 06:35:24 dist-test-slave-6zbq krb5kdc[7590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998124, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.43.125@KRBTEST.COM
W20260806 06:35:24.898684 7942 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4022cbe9ccd743ea9384daf248e44eda: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:35:24.898728 7942 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:35:24.900071 7820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:24.892970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40674 (local address 127.7.43.125:37521)
0806 06:35:24.893001 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:24.893003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:24.893013 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:35:24.893070 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:24.893070 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:24.893077 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:24.893127 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:24.893707 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:24.894255 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:24.895148 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:24.895343 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:24.895394 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:24.897555 (+ 2161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:24.897556 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:24.897557 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:24.897563 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:24.899093 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:24.899435 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:24.899436 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:24.899437 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:24.899491 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:24.899648 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:24.899649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:24.899649 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:24.899753 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:24.899897 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:24.899932 (+ 35us) server_negotiation.cc:299] Negotiation successful
0806 06:35:24.899993 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260806 06:35:24.901221 7341 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:35:24.899338 7823 master_runner.cc:186] Error getting master registration for 127.7.43.126:43259: OK, instance_id { permanent_uuid: "3024873d038b4cad85afce8303eff7b4" instance_seqno: 1785998124570423 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 06:35:24.901379 7823 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 06:35:24.902154 7667 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "3024873d038b4cad85afce8303eff7b4"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59134
I20260806 06:35:24.909482 7711 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.43.124" port: 46385 } attempt: 2
Aug 06 06:35:24 dist-test-slave-6zbq krb5kdc[7590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998124, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.126@KRBTEST.COM for kudu/127.7.43.124@KRBTEST.COM
I20260806 06:35:24.916250 7927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:24.909788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53958 (local address 127.7.43.124:46385)
0806 06:35:24.909837 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:24.909838 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:24.909846 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:35:24.909865 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:24.909866 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:24.909874 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:24.909922 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:24.910179 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:24.910921 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:24.911830 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:24.912155 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:24.912214 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:24.913903 (+ 1689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:24.913904 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:24.913904 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:24.913911 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:24.915207 (+ 1296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:24.915561 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:24.915561 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:24.915562 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:24.915612 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:24.915779 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:24.915779 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:24.915780 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:24.915918 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:24.915976 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:24.916097 (+ 121us) server_negotiation.cc:299] Negotiation successful
0806 06:35:24.916168 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 06:35:24.916260 7714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:24.909716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.124:46385 (local address 127.0.0.1:53958)
0806 06:35:24.909756 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:24.909767 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:24.909810 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:24.909939 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:24.909939 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:24.910084 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:35:24.910120 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:24.910122 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:24.910985 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:24.910986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:24.911761 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:24.911762 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:24.912225 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:24.912225 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:24.912280 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:24.912605 (+ 325us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:35:24.912609 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:35:24.913827 (+ 1218us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:35:24.915294 (+ 1467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:24.915295 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:24.915295 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:24.915501 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:24.915669 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:24.915670 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:24.915670 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:24.915728 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:24.915974 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:35:24.915974 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:35:24.916016 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 06:35:24.916079 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 06:35:24.916171 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260806 06:35:24.917601 7711 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3024873d038b4cad85afce8303eff7b4: Bootstrap starting.
I20260806 06:35:24.918198 7711 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3024873d038b4cad85afce8303eff7b4: Neither blocks nor log segments found. Creating new log.
I20260806 06:35:24.918494 7711 log.cc:824] T 00000000000000000000000000000000 P 3024873d038b4cad85afce8303eff7b4: Log is configured to *not* fsync() on all Append() calls
I20260806 06:35:24.919126 7711 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3024873d038b4cad85afce8303eff7b4: No bootstrap required, opened a new log
I20260806 06:35:24.920063 7711 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3024873d038b4cad85afce8303eff7b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3024873d038b4cad85afce8303eff7b4" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 43259 } } peers { permanent_uuid: "a8b6a480861246abb83589b127e50c0a" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 37521 } } peers { permanent_uuid: "4022cbe9ccd743ea9384daf248e44eda" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 46385 } }
I20260806 06:35:24.920164 7711 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3024873d038b4cad85afce8303eff7b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3024873d038b4cad85afce8303eff7b4, State: Initialized, Role: FOLLOWER
I20260806 06:35:24.920233 7711 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3024873d038b4cad85afce8303eff7b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3024873d038b4cad85afce8303eff7b4" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 43259 } } peers { permanent_uuid: "a8b6a480861246abb83589b127e50c0a" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 37521 } } peers { permanent_uuid: "4022cbe9ccd743ea9384daf248e44eda" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 46385 } }
I20260806 06:35:24.920471 7946 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3024873d038b4cad85afce8303eff7b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3024873d038b4cad85afce8303eff7b4" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 43259 } } peers { permanent_uuid: "a8b6a480861246abb83589b127e50c0a" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 37521 } } peers { permanent_uuid: "4022cbe9ccd743ea9384daf248e44eda" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 46385 } } }
I20260806 06:35:24.920516 7711 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3024873d038b4cad85afce8303eff7b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:35:24.920545 7667 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3024873d038b4cad85afce8303eff7b4 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 06:35:24.920642 7667 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3024873d038b4cad85afce8303eff7b4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:35:24.921409 7667 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3024873d038b4cad85afce8303eff7b4 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3024873d038b4cad85afce8303eff7b4" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 43259 } } peers { permanent_uuid: "a8b6a480861246abb83589b127e50c0a" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 37521 } } peers { permanent_uuid: "4022cbe9ccd743ea9384daf248e44eda" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 46385 } }
I20260806 06:35:24.921680 7946 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3024873d038b4cad85afce8303eff7b4 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:35:24.921765 7667 leader_election.cc:288] T 00000000000000000000000000000000 P 3024873d038b4cad85afce8303eff7b4 [CANDIDATE]: Term 1 election: Requested vote from peers a8b6a480861246abb83589b127e50c0a (127.7.43.125:37521), 4022cbe9ccd743ea9384daf248e44eda (127.7.43.124:46385)
I20260806 06:35:24.921902 7882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3024873d038b4cad85afce8303eff7b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "4022cbe9ccd743ea9384daf248e44eda"
I20260806 06:35:24.922000 7882 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4022cbe9ccd743ea9384daf248e44eda [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:35:24.922991 7882 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4022cbe9ccd743ea9384daf248e44eda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3024873d038b4cad85afce8303eff7b4 in term 1.
I20260806 06:35:24.924155 7628 leader_election.cc:302] T 00000000000000000000000000000000 P 3024873d038b4cad85afce8303eff7b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3024873d038b4cad85afce8303eff7b4, 4022cbe9ccd743ea9384daf248e44eda; no voters:
I20260806 06:35:24.924413 7946 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3024873d038b4cad85afce8303eff7b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:35:24.924551 7946 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3024873d038b4cad85afce8303eff7b4 [term 1 LEADER]: Becoming Leader. State: Replica: 3024873d038b4cad85afce8303eff7b4, State: Running, Role: LEADER
I20260806 06:35:24.924640 7946 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3024873d038b4cad85afce8303eff7b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3024873d038b4cad85afce8303eff7b4" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 43259 } } peers { permanent_uuid: "a8b6a480861246abb83589b127e50c0a" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 37521 } } peers { permanent_uuid: "4022cbe9ccd743ea9384daf248e44eda" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 46385 } }
I20260806 06:35:24.925031 7946 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3024873d038b4cad85afce8303eff7b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3024873d038b4cad85afce8303eff7b4. Latest consensus state: current_term: 1 leader_uuid: "3024873d038b4cad85afce8303eff7b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3024873d038b4cad85afce8303eff7b4" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 43259 } } peers { permanent_uuid: "a8b6a480861246abb83589b127e50c0a" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 37521 } } peers { permanent_uuid: "4022cbe9ccd743ea9384daf248e44eda" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 46385 } } }
I20260806 06:35:24.925105 7946 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3024873d038b4cad85afce8303eff7b4 [sys.catalog]: This master's current role is: LEADER
W20260806 06:35:24.925184 7969 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3024873d038b4cad85afce8303eff7b4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260806 06:35:24.925945 7606 master_runner.cc:186] Error getting master registration for 127.7.43.125:37521: OK, instance_id { permanent_uuid: "a8b6a480861246abb83589b127e50c0a" instance_seqno: 1785998124714992 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 06:35:24.928062 7606 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 06:35:24.928159 7969 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:35:24.933351 7927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:24.923301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53968 (local address 127.7.43.124:46385)
0806 06:35:24.923377 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:24.923379 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:24.923770 (+ 391us) server_negotiation.cc:407] Connection header received
0806 06:35:24.923909 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:24.923910 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:24.923931 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:24.923957 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:24.925065 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:24.925760 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:24.927302 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:24.927586 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:24.927625 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:24.929891 (+ 2266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:24.929892 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:24.929892 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:24.929910 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:24.931581 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:24.932170 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:24.932172 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:24.932172 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:24.932228 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:24.932618 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:24.932619 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:24.932619 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:24.932793 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:24.932879 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:24.933196 (+ 317us) server_negotiation.cc:299] Negotiation successful
0806 06:35:24.933270 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 06:35:24.933930 7715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:24.923405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59136 (local address 127.7.43.126:43259)
0806 06:35:24.924300 (+ 895us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:24.924301 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:24.924310 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:35:24.924325 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:24.924326 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:24.924335 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:24.924382 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:24.925917 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:24.926569 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:24.927512 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:24.927784 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:24.927823 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:24.931212 (+ 3389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:24.931213 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:24.931213 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:24.931220 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:24.932717 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:24.933141 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:24.933142 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:24.933142 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:24.933198 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:24.933396 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:24.933397 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:24.933397 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:24.933516 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:24.933575 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:24.933787 (+ 212us) server_negotiation.cc:299] Negotiation successful
0806 06:35:24.933856 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 06:35:24.935961 7976 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:35:24.936165 7976 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:35:24.937834 7882 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4022cbe9ccd743ea9384daf248e44eda [term 1 FOLLOWER]: Refusing update from remote peer 3024873d038b4cad85afce8303eff7b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:35:24.938110 7946 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3024873d038b4cad85afce8303eff7b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4022cbe9ccd743ea9384daf248e44eda" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 46385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:35:24.939320 7929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4022cbe9ccd743ea9384daf248e44eda [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3024873d038b4cad85afce8303eff7b4. Latest consensus state: current_term: 1 leader_uuid: "3024873d038b4cad85afce8303eff7b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3024873d038b4cad85afce8303eff7b4" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 43259 } } peers { permanent_uuid: "a8b6a480861246abb83589b127e50c0a" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 37521 } } peers { permanent_uuid: "4022cbe9ccd743ea9384daf248e44eda" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 46385 } } }
I20260806 06:35:24.939399 7929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4022cbe9ccd743ea9384daf248e44eda [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:35:24.942498 7820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:24.923595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40688 (local address 127.7.43.125:37521)
0806 06:35:24.923630 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:24.923632 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:24.923909 (+ 277us) server_negotiation.cc:407] Connection header received
0806 06:35:24.923990 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:24.923991 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:24.923999 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:24.924026 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:24.928876 (+ 4850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:24.929528 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:24.934617 (+ 5089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:24.935543 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:24.935588 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:24.939025 (+ 3437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:24.939026 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:24.939027 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:24.939034 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:24.940842 (+ 1808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:24.941448 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:24.941449 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:24.941450 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:24.941506 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:24.941795 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:24.941796 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:24.941796 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:24.941930 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:24.942002 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:24.942281 (+ 279us) server_negotiation.cc:299] Negotiation successful
0806 06:35:24.942394 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:35:24.946326 7816 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.43.124" port: 46385 } attempt: 3
I20260806 06:35:24.948640 7976 catalog_manager.cc:1347] Generated new cluster ID: 0504e40d02c246be9820f12efa88e07d
I20260806 06:35:24.948763 7976 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:35:24 dist-test-slave-6zbq krb5kdc[7590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998124, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.125@KRBTEST.COM for kudu/127.7.43.124@KRBTEST.COM
I20260806 06:35:24.954674 7818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:24.946570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.124:46385 (local address 127.0.0.1:53970)
0806 06:35:24.946614 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:24.946624 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:24.946656 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:24.946967 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:24.946968 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:24.947123 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:35:24.947161 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:24.947163 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:24.948029 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:24.948030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:24.948816 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:24.948819 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:24.949303 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:24.949304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:24.949360 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:24.949726 (+ 366us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:35:24.949731 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:35:24.951108 (+ 1377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:35:24.953651 (+ 2543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:24.953652 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:24.953652 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:24.953860 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:24.954056 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:24.954056 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:24.954057 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:24.954122 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:24.954399 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:35:24.954400 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:35:24.954441 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 06:35:24.954500 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 06:35:24.954585 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 06:35:24.955037 7927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:24.946721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53970 (local address 127.7.43.124:46385)
0806 06:35:24.946775 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:24.946777 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:24.946785 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:35:24.946804 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:24.946805 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:24.946814 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:24.946868 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:24.947236 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:24.947885 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:24.948908 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:24.949202 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:24.949255 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:24.951209 (+ 1954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:24.951211 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:24.951211 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:24.951217 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:24.953552 (+ 2335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:24.953940 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:24.953941 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:24.953941 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:24.953992 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:24.954186 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:24.954187 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:24.954188 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:24.954294 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:24.954374 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:24.954869 (+ 495us) server_negotiation.cc:299] Negotiation successful
0806 06:35:24.954937 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 06:35:24.956357 7816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8b6a480861246abb83589b127e50c0a: Bootstrap starting.
I20260806 06:35:24.957008 7816 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a8b6a480861246abb83589b127e50c0a: Neither blocks nor log segments found. Creating new log.
I20260806 06:35:24.957302 7816 log.cc:824] T 00000000000000000000000000000000 P a8b6a480861246abb83589b127e50c0a: Log is configured to *not* fsync() on all Append() calls
I20260806 06:35:24.958210 7977 mvcc.cc:205] Tried to move back new op lower bound from 7315448319742799872 to 7315448319692017664. Current Snapshot: MvccSnapshot[applied={T|T < 7315448319742799872 or (T in {7315448319742799872})}]
I20260806 06:35:24.958567 7946 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3024873d038b4cad85afce8303eff7b4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3024873d038b4cad85afce8303eff7b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3024873d038b4cad85afce8303eff7b4" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 43259 } } peers { permanent_uuid: "a8b6a480861246abb83589b127e50c0a" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 37521 } } peers { permanent_uuid: "4022cbe9ccd743ea9384daf248e44eda" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 46385 } } }
I20260806 06:35:24.958652 7946 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3024873d038b4cad85afce8303eff7b4 [sys.catalog]: This master's current role is: LEADER
I20260806 06:35:24.959107 7929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4022cbe9ccd743ea9384daf248e44eda [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3024873d038b4cad85afce8303eff7b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3024873d038b4cad85afce8303eff7b4" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 43259 } } peers { permanent_uuid: "a8b6a480861246abb83589b127e50c0a" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 37521 } } peers { permanent_uuid: "4022cbe9ccd743ea9384daf248e44eda" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 46385 } } }
I20260806 06:35:24.959185 7929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4022cbe9ccd743ea9384daf248e44eda [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:35:24.962848 7816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8b6a480861246abb83589b127e50c0a: No bootstrap required, opened a new log
I20260806 06:35:24.964761 7816 raft_consensus.cc:348] T 00000000000000000000000000000000 P a8b6a480861246abb83589b127e50c0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3024873d038b4cad85afce8303eff7b4" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 43259 } } peers { permanent_uuid: "a8b6a480861246abb83589b127e50c0a" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 37521 } } peers { permanent_uuid: "4022cbe9ccd743ea9384daf248e44eda" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 46385 } }
I20260806 06:35:24.964890 7816 raft_consensus.cc:729] T 00000000000000000000000000000000 P a8b6a480861246abb83589b127e50c0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8b6a480861246abb83589b127e50c0a, State: Initialized, Role: FOLLOWER
I20260806 06:35:24.964962 7816 consensus_queue.cc:260] T 00000000000000000000000000000000 P a8b6a480861246abb83589b127e50c0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3024873d038b4cad85afce8303eff7b4" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 43259 } } peers { permanent_uuid: "a8b6a480861246abb83589b127e50c0a" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 37521 } } peers { permanent_uuid: "4022cbe9ccd743ea9384daf248e44eda" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 46385 } }
I20260806 06:35:24.965271 7816 sys_catalog.cc:564] T 00000000000000000000000000000000 P a8b6a480861246abb83589b127e50c0a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:35:24.965356 7779 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a8b6a480861246abb83589b127e50c0a [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:35:24.966192 7779 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a8b6a480861246abb83589b127e50c0a [term 1 FOLLOWER]: Refusing update from remote peer 3024873d038b4cad85afce8303eff7b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:35:24.966543 7781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3024873d038b4cad85afce8303eff7b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a8b6a480861246abb83589b127e50c0a"
I20260806 06:35:24.967281 7781 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a8b6a480861246abb83589b127e50c0a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3024873d038b4cad85afce8303eff7b4 in term 1.
I20260806 06:35:24.967331 7983 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3024873d038b4cad85afce8303eff7b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8b6a480861246abb83589b127e50c0a" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 37521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:35:24.969043 7986 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8b6a480861246abb83589b127e50c0a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3024873d038b4cad85afce8303eff7b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3024873d038b4cad85afce8303eff7b4" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 43259 } } peers { permanent_uuid: "a8b6a480861246abb83589b127e50c0a" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 37521 } } peers { permanent_uuid: "4022cbe9ccd743ea9384daf248e44eda" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 46385 } } }
I20260806 06:35:24.969194 7986 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8b6a480861246abb83589b127e50c0a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:35:24.969288 7986 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8b6a480861246abb83589b127e50c0a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3024873d038b4cad85afce8303eff7b4. Latest consensus state: current_term: 1 leader_uuid: "3024873d038b4cad85afce8303eff7b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3024873d038b4cad85afce8303eff7b4" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 43259 } } peers { permanent_uuid: "a8b6a480861246abb83589b127e50c0a" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 37521 } } peers { permanent_uuid: "4022cbe9ccd743ea9384daf248e44eda" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 46385 } } }
I20260806 06:35:24.969378 7986 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8b6a480861246abb83589b127e50c0a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:35:24.969446 7986 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8b6a480861246abb83589b127e50c0a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3024873d038b4cad85afce8303eff7b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3024873d038b4cad85afce8303eff7b4" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 43259 } } peers { permanent_uuid: "a8b6a480861246abb83589b127e50c0a" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 37521 } } peers { permanent_uuid: "4022cbe9ccd743ea9384daf248e44eda" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 46385 } } }
I20260806 06:35:24.969529 7986 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8b6a480861246abb83589b127e50c0a [sys.catalog]: This master's current role is: FOLLOWER
W20260806 06:35:24.970021 8009 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a8b6a480861246abb83589b127e50c0a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:35:24.970108 8009 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:35:24.970841 7946 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3024873d038b4cad85afce8303eff7b4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3024873d038b4cad85afce8303eff7b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3024873d038b4cad85afce8303eff7b4" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 43259 } } peers { permanent_uuid: "a8b6a480861246abb83589b127e50c0a" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 37521 } } peers { permanent_uuid: "4022cbe9ccd743ea9384daf248e44eda" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 46385 } } }
I20260806 06:35:24.970916 7946 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3024873d038b4cad85afce8303eff7b4 [sys.catalog]: This master's current role is: LEADER
I20260806 06:35:24.971410 7976 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:35:24.972122 7976 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:35:24.978269 8004 mvcc.cc:205] Tried to move back new op lower bound from 7315448319742799872 to 7315448319692017664. Current Snapshot: MvccSnapshot[applied={T|T < 7315448319742799872}]
I20260806 06:35:24.979100 7976 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3024873d038b4cad85afce8303eff7b4: Generated new TSK 0
I20260806 06:35:24.979323 7976 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 06:35:25.012051 7715 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:35:25.010831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59148 (local address 127.7.43.126:43259)
0806 06:35:25.010869 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:25.010871 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:25.011466 (+ 595us) server_negotiation.cc:407] Connection header received
0806 06:35:25.011561 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:25.011562 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:25.011572 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:25.011599 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:25.011993 (+ 394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59148: BlockingRecv error: recv got EOF from 127.0.0.1:59148 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260806 06:35:25.012732 7820 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:35:25.011178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40694 (local address 127.7.43.125:37521)
0806 06:35:25.011212 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:25.011214 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:25.012281 (+ 1067us) server_negotiation.cc:407] Connection header received
0806 06:35:25.012368 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:25.012369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:25.012378 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:25.012405 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:25.012677 (+ 272us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40694: BlockingRecv error: recv got EOF from 127.0.0.1:40694 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260806 06:35:25.013703 7927 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:35:25.013042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53986 (local address 127.7.43.124:46385)
0806 06:35:25.013074 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:25.013076 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:25.013204 (+ 128us) server_negotiation.cc:407] Connection header received
0806 06:35:25.013302 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:25.013303 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:25.013312 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:25.013339 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:25.013650 (+ 311us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53986: BlockingRecv error: recv got EOF from 127.0.0.1:53986 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260806 06:35:25.015715 7820 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:35:25.015035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40708 (local address 127.7.43.125:37521)
0806 06:35:25.015065 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:25.015066 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:25.015261 (+ 195us) server_negotiation.cc:407] Connection header received
0806 06:35:25.015355 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:25.015356 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:25.015370 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:25.015395 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:25.015675 (+ 280us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40708: BlockingRecv error: recv got EOF from 127.0.0.1:40708 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260806 06:35:25.018448 7820 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:35:25.017728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40722 (local address 127.7.43.125:37521)
0806 06:35:25.017758 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:25.017759 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:25.017975 (+ 216us) server_negotiation.cc:407] Connection header received
0806 06:35:25.018062 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:25.018063 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:25.018072 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:25.018097 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:25.018407 (+ 310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40722: BlockingRecv error: recv got EOF from 127.0.0.1:40722 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260806 06:35:25.022195 7820 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:35:25.021463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40724 (local address 127.7.43.125:37521)
0806 06:35:25.021491 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:25.021492 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:25.021735 (+ 243us) server_negotiation.cc:407] Connection header received
0806 06:35:25.021852 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:25.021853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:25.021862 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:25.021888 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:25.022157 (+ 269us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40724: BlockingRecv error: recv got EOF from 127.0.0.1:40724 (error 108)
Metrics: {"server-negotiator.queue_time_us":14}
W20260806 06:35:25.027853 7820 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:35:25.027157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40740 (local address 127.7.43.125:37521)
0806 06:35:25.027188 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:25.027190 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:25.027413 (+ 223us) server_negotiation.cc:407] Connection header received
0806 06:35:25.027499 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:25.027500 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:25.027509 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:25.027535 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:25.027812 (+ 277us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40740: BlockingRecv error: recv got EOF from 127.0.0.1:40740 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260806 06:35:25.037575 7820 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:35:25.036839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40748 (local address 127.7.43.125:37521)
0806 06:35:25.036872 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:25.036874 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:25.037098 (+ 224us) server_negotiation.cc:407] Connection header received
0806 06:35:25.037182 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:25.037183 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:25.037192 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:25.037218 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:25.037528 (+ 310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40748: BlockingRecv error: recv got EOF from 127.0.0.1:40748 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260806 06:35:25.055438 7820 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:35:25.054606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40752 (local address 127.7.43.125:37521)
0806 06:35:25.054638 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:25.054640 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:25.054918 (+ 278us) server_negotiation.cc:407] Connection header received
0806 06:35:25.055022 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:25.055023 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:25.055033 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:25.055060 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:25.055397 (+ 337us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40752: BlockingRecv error: recv got EOF from 127.0.0.1:40752 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260806 06:35:25.089303 7820 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:35:25.088574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40754 (local address 127.7.43.125:37521)
0806 06:35:25.088620 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:25.088621 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:25.088772 (+ 151us) server_negotiation.cc:407] Connection header received
0806 06:35:25.088827 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:25.088828 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:25.088839 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:25.088899 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:25.089246 (+ 347us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40754: BlockingRecv error: recv got EOF from 127.0.0.1:40754 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260806 06:35:25.155030 7820 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:35:25.154314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40756 (local address 127.7.43.125:37521)
0806 06:35:25.154400 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:25.154402 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:25.154470 (+ 68us) server_negotiation.cc:407] Connection header received
0806 06:35:25.154532 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:25.154533 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:25.154543 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:25.154596 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:25.154972 (+ 376us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40756: BlockingRecv error: recv got EOF from 127.0.0.1:40756 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260806 06:35:25.284686 7820 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:35:25.284020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40772 (local address 127.7.43.125:37521)
0806 06:35:25.284091 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:25.284093 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:25.284175 (+ 82us) server_negotiation.cc:407] Connection header received
0806 06:35:25.284249 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:25.284250 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:25.284262 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:25.284313 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:25.284624 (+ 311us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40772: BlockingRecv error: recv got EOF from 127.0.0.1:40772 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260806 06:35:25.542282 7820 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:35:25.541677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40780 (local address 127.7.43.125:37521)
0806 06:35:25.541740 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:25.541742 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:25.541751 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:35:25.541836 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:25.541837 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:25.541848 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:25.541897 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:25.542212 (+ 315us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40780: BlockingRecv error: recv got EOF from 127.0.0.1:40780 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 06:35:25.899111 7942 catalog_manager.cc:1259] Loaded cluster ID: 0504e40d02c246be9820f12efa88e07d
I20260806 06:35:25.899173 7942 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4022cbe9ccd743ea9384daf248e44eda: loading cluster ID for follower catalog manager: success
I20260806 06:35:25.899930 7942 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4022cbe9ccd743ea9384daf248e44eda: acquiring CA information for follower catalog manager: success
I20260806 06:35:25.900210 7942 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4022cbe9ccd743ea9384daf248e44eda: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 06:35:25.970542 8009 catalog_manager.cc:1259] Loaded cluster ID: 0504e40d02c246be9820f12efa88e07d
I20260806 06:35:25.970600 8009 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a8b6a480861246abb83589b127e50c0a: loading cluster ID for follower catalog manager: success
I20260806 06:35:25.971385 8009 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a8b6a480861246abb83589b127e50c0a: acquiring CA information for follower catalog manager: success
I20260806 06:35:25.971679 8009 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a8b6a480861246abb83589b127e50c0a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 06:35:26.057513 8103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:26.055312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39220 (local address 127.7.43.125:37521)
0806 06:35:26.055422 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:26.055424 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:26.055439 (+ 15us) server_negotiation.cc:407] Connection header received
0806 06:35:26.055499 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:26.055500 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:26.055544 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:26.055614 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:35:26.055728 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:26.056207 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:26.056917 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:26.057105 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:26.057159 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:26.057265 (+ 106us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:35:26.057367 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:35:26.057421 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:26.057436 (+ 15us) server_negotiation.cc:299] Negotiation successful
0806 06:35:26.057453 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":55,"threads_started":1}
I20260806 06:35:26.061328 8111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:26.059143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42790 (local address 127.7.43.124:46385)
0806 06:35:26.059262 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:26.059264 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:26.059271 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:35:26.059309 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:26.059309 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:26.059342 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:26.059404 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:35:26.059502 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:26.059969 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:26.060650 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:26.060841 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:26.060893 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:26.061025 (+ 132us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:35:26.061125 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:35:26.061185 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:26.061232 (+ 47us) server_negotiation.cc:299] Negotiation successful
0806 06:35:26.061251 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":56,"threads_started":1}
I20260806 06:35:26.065434 8111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:26.062948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42796 (local address 127.7.43.124:46385)
0806 06:35:26.062990 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:26.062992 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:26.062998 (+ 6us) server_negotiation.cc:407] Connection header received
0806 06:35:26.063016 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:26.063017 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:26.063026 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:26.063069 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:35:26.063204 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:26.063702 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:26.064666 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:26.064918 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:26.064975 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:26.065096 (+ 121us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:35:26.065122 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:35:26.065166 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:26.065355 (+ 189us) server_negotiation.cc:299] Negotiation successful
0806 06:35:26.065376 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:35:26.065515 8103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:26.062768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39236 (local address 127.7.43.125:37521)
0806 06:35:26.062814 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:26.062816 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:26.063010 (+ 194us) server_negotiation.cc:407] Connection header received
0806 06:35:26.063065 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:26.063066 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:26.063074 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:26.063144 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:35:26.063237 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:26.063917 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:26.064823 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:26.065049 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:26.065103 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:26.065308 (+ 205us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:35:26.065338 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:35:26.065382 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:26.065433 (+ 51us) server_negotiation.cc:299] Negotiation successful
0806 06:35:26.065451 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 06:35:26.065842 8118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:26.062784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57658 (local address 127.7.43.126:43259)
0806 06:35:26.062972 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:26.062974 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:26.063118 (+ 144us) server_negotiation.cc:407] Connection header received
0806 06:35:26.063165 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:26.063166 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:26.063213 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:26.063287 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:35:26.063386 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:26.064087 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:26.064837 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:26.065169 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:26.065223 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:26.065392 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:35:26.065486 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:35:26.065546 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:26.065735 (+ 189us) server_negotiation.cc:299] Negotiation successful
0806 06:35:26.065763 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":53,"threads_started":1}
I20260806 06:35:26.067224 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 7606
I20260806 06:35:26.080417 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 7720
I20260806 06:35:26.088018 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 7823
2026-08-06T06:35:26Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2842 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:35:26 dist-test-slave-6zbq krb5kdc[8124](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:35:26 dist-test-slave-6zbq krb5kdc[8124](info): set up 2 sockets
Aug 06 06:35:26 dist-test-slave-6zbq krb5kdc[8124](info): commencing operation
krb5kdc: starting...
W20260806 06:35:27.189373 7341 mini_kdc.cc:121] Time spent starting KDC: real 1.080s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/krb5kdc/test-user.keytab.
Aug 06 06:35:27 dist-test-slave-6zbq krb5kdc[8124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:35:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:35:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.43.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
Entry for principal kudu/127.7.43.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab.
I20260806 06:35:27.303421 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:37427
--webserver_interface=127.7.43.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:46317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.43.126:37427,127.7.43.125:44543,127.7.43.124:45495
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:35:27.383750 8140 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 06:35:27.383952 8140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:35:27.383978 8140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:35:27.384002 8140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:35:27.385088 8140 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:35:27.385134 8140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:35:27.385154 8140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:35:27.385169 8140 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:35:27.385185 8140 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:35:27.386664 8140 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:46317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.43.126:37427,127.7.43.125:44543,127.7.43.124:45495
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.126.keytab
--principal=kudu/127.7.43.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.126:37427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.43.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8140
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:35:27.386881 8140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:35:27.387081 8140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:35:27.391503 8149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:27.391589 8140 server_base.cc:1034] running on GCE node
W20260806 06:35:27.391521 8148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:27.391521 8152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:27.391899 8140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:35:27.392167 8140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:35:27.393324 8140 hybrid_clock.cc:648] HybridClock initialized: now 1785998127393307 us; error 34 us; skew 500 ppm
Aug 06 06:35:27 dist-test-slave-6zbq krb5kdc[8124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998127, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:35:27.399715 8140 init.cc:377] Logged in from keytab as kudu/127.7.43.126@KRBTEST.COM (short username kudu)
I20260806 06:35:27.400121 8140 webserver.cc:460] Webserver started at http://127.7.43.126:39259/ using document root <none> and password file <none>
I20260806 06:35:27.400326 8140 fs_manager.cc:359] Metadata directory not provided
I20260806 06:35:27.400373 8140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:35:27.400493 8140 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:35:27.401446 8140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/minicluster-data/master-0/data/instance:
uuid: "69f8b2f4d314472eb4dde3c0d5ac8351"
format_stamp: "Formatted at 2026-08-06 06:35:27 on dist-test-slave-6zbq"
server_key: "e1c46f7aa986b1a8ed2f6c6e7962aa9c"
server_key_iv: "62d2e9990102417f75931a1f578f6612"
server_key_version: "encryptionkey@0"
I20260806 06:35:27.401772 8140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/minicluster-data/master-0/wal/instance:
uuid: "69f8b2f4d314472eb4dde3c0d5ac8351"
format_stamp: "Formatted at 2026-08-06 06:35:27 on dist-test-slave-6zbq"
server_key: "e1c46f7aa986b1a8ed2f6c6e7962aa9c"
server_key_iv: "62d2e9990102417f75931a1f578f6612"
server_key_version: "encryptionkey@0"
I20260806 06:35:27.402990 8140 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:35:27.403595 8161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:27.403741 8140 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:35:27.403822 8140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/minicluster-data/master-0/wal
uuid: "69f8b2f4d314472eb4dde3c0d5ac8351"
format_stamp: "Formatted at 2026-08-06 06:35:27 on dist-test-slave-6zbq"
server_key: "e1c46f7aa986b1a8ed2f6c6e7962aa9c"
server_key_iv: "62d2e9990102417f75931a1f578f6612"
server_key_version: "encryptionkey@0"
I20260806 06:35:27.403882 8140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:35:27.413829 8140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:35:27.415799 8140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:35:27.415922 8140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:35:27.419564 8140 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.126:37427
I20260806 06:35:27.419592 8230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.126:37427 every 8 connection(s)
I20260806 06:35:27.419883 8140 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/minicluster-data/master-0/data/info.pb
I20260806 06:35:27.420737 8231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:35:27.422063 8231 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.43.126" port: 37427 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:35:27 dist-test-slave-6zbq krb5kdc[8124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998127, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.126@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:35:27.427947 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 8140
I20260806 06:35:27.428051 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/minicluster-data/master-0/wal/instance
I20260806 06:35:27.428236 7341 external_mini_cluster.cc:1428] Setting key cbee455083ac9b82c7054644534880b6
I20260806 06:35:27.430605 8234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:27.422650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:37427 (local address 127.0.0.1:33598)
0806 06:35:27.422815 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:27.422845 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:27.423240 (+ 395us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:27.423489 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:27.423491 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:27.423696 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:35:27.423850 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:27.423854 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:27.424823 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:27.424824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:27.425461 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:27.425463 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:27.425900 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:27.425900 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:27.425949 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:27.426307 (+ 358us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:35:27.426314 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:35:27.427565 (+ 1251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:35:27.429193 (+ 1628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:27.429201 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:27.429203 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:27.429444 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:27.429647 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:27.429648 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:27.429648 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:27.429754 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:27.430182 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:35:27.430186 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:35:27.430238 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 06:35:27.430312 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 06:35:27.430462 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":89,"threads_started":1}
I20260806 06:35:27.430675 8235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:27.422687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33598 (local address 127.7.43.126:37427)
0806 06:35:27.422877 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:27.422881 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:27.422905 (+ 24us) server_negotiation.cc:407] Connection header received
0806 06:35:27.423386 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:27.423389 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:27.423405 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:27.423477 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:27.423951 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:27.424748 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:27.425531 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:27.425828 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:27.425900 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:27.427637 (+ 1737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:27.427645 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:27.427645 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:27.427651 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:27.429111 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:27.429512 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:27.429514 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:27.429514 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:27.429584 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:27.429840 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:27.429842 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:27.429842 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:27.430116 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:27.430197 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:27.430385 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 06:35:27.430554 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":69,"server-negotiator.queue_time_us":169,"thread_start_us":67,"threads_started":1}
I20260806 06:35:27.431277 8231 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.43.125" port: 44543 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:35:27.431555 8234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:27.431463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.125:44543 (local address 127.0.0.1:58006)
0806 06:35:27.431500 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:27.431537 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.125:44543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260806 06:35:27.431687 8162 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.43.125:44543: connect: Connection refused (error 111)
I20260806 06:35:27.432065 8234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:27.431988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.125:44543 (local address 127.0.0.1:58010)
0806 06:35:27.432026 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:27.432049 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.125:44543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260806 06:35:27.432156 8231 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.43.125:44543: Network error: Client connection negotiation failed: client connection to 127.7.43.125:44543: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.7.43.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.125.keytab.
Entry for principal kudu/127.7.43.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.125.keytab.
I20260806 06:35:27.450073 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.125:44543
--webserver_interface=127.7.43.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.125.keytab
--principal=kudu/127.7.43.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:46317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.43.126:37427,127.7.43.125:44543,127.7.43.124:45495
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:35:27.481392 8231 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.43.125" port: 44543 } attempt: 1
I20260806 06:35:27.481801 8234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:27.481696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.125:44543 (local address 127.0.0.1:58026)
0806 06:35:27.481748 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:27.481776 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.125:44543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 06:35:27.482146 8234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:27.482012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.125:44543 (local address 127.0.0.1:58028)
0806 06:35:27.482063 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:27.482120 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.125:44543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 06:35:27.482244 8231 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.43.125:44543: Network error: Client connection negotiation failed: client connection to 127.7.43.125:44543: connect: Connection refused (error 111)
W20260806 06:35:27.527716 8240 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 06:35:27.527894 8240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:35:27.527910 8240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:35:27.527928 8240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:35:27.529006 8240 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:35:27.529045 8240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:35:27.529057 8240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:35:27.529080 8240 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:35:27.529103 8240 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:35:27.530555 8240 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:46317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.43.126:37427,127.7.43.125:44543,127.7.43.124:45495
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.125.keytab
--principal=kudu/127.7.43.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.125:44543
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.43.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:35:27.530758 8240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:35:27.530942 8240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:35:27.535208 8246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:27.535202 8245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:27.535197 8248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:27.535307 8240 server_base.cc:1034] running on GCE node
I20260806 06:35:27.535599 8240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:35:27.535861 8240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:35:27.537005 8240 hybrid_clock.cc:648] HybridClock initialized: now 1785998127536990 us; error 29 us; skew 500 ppm
Aug 06 06:35:27 dist-test-slave-6zbq krb5kdc[8124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998127, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:35:27.539222 8240 init.cc:377] Logged in from keytab as kudu/127.7.43.125@KRBTEST.COM (short username kudu)
I20260806 06:35:27.539584 8240 webserver.cc:460] Webserver started at http://127.7.43.125:46553/ using document root <none> and password file <none>
I20260806 06:35:27.539781 8240 fs_manager.cc:359] Metadata directory not provided
I20260806 06:35:27.539827 8240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:35:27.539947 8240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:35:27.540860 8240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/minicluster-data/master-1/data/instance:
uuid: "9cc0b4cc918048c5a7e743771b1b5f97"
format_stamp: "Formatted at 2026-08-06 06:35:27 on dist-test-slave-6zbq"
server_key: "f1c8673bc31e71f9453616dbda0c9e2f"
server_key_iv: "21d8cd4226731303408a4f931df9d3e2"
server_key_version: "encryptionkey@0"
I20260806 06:35:27.541180 8240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/minicluster-data/master-1/wal/instance:
uuid: "9cc0b4cc918048c5a7e743771b1b5f97"
format_stamp: "Formatted at 2026-08-06 06:35:27 on dist-test-slave-6zbq"
server_key: "f1c8673bc31e71f9453616dbda0c9e2f"
server_key_iv: "21d8cd4226731303408a4f931df9d3e2"
server_key_version: "encryptionkey@0"
I20260806 06:35:27.542377 8240 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:35:27.543040 8260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:27.543202 8240 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:35:27.543282 8240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/minicluster-data/master-1/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/minicluster-data/master-1/wal
uuid: "9cc0b4cc918048c5a7e743771b1b5f97"
format_stamp: "Formatted at 2026-08-06 06:35:27 on dist-test-slave-6zbq"
server_key: "f1c8673bc31e71f9453616dbda0c9e2f"
server_key_iv: "21d8cd4226731303408a4f931df9d3e2"
server_key_version: "encryptionkey@0"
I20260806 06:35:27.543346 8240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:35:27.550395 8231 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.43.125" port: 44543 } attempt: 2
I20260806 06:35:27.550711 8234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:27.550612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.125:44543 (local address 127.0.0.1:58032)
0806 06:35:27.550660 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:27.550688 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.125:44543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260806 06:35:27.551007 8234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:27.550926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.125:44543 (local address 127.0.0.1:58044)
0806 06:35:27.550966 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:27.550988 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.125:44543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260806 06:35:27.551080 8231 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.43.125:44543: Network error: Client connection negotiation failed: client connection to 127.7.43.125:44543: connect: Connection refused (error 111)
I20260806 06:35:27.558241 8240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:35:27.560194 8240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:35:27.560316 8240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:35:27.563912 8240 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.125:44543
I20260806 06:35:27.563949 8340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.125:44543 every 8 connection(s)
I20260806 06:35:27.564221 8240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/minicluster-data/master-1/data/info.pb
I20260806 06:35:27.565088 8341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:35:27.566430 8341 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.43.126" port: 37427 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:35:27 dist-test-slave-6zbq krb5kdc[8124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998127, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.125@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:35:27.573654 8235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:27.566999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33600 (local address 127.7.43.126:37427)
0806 06:35:27.567049 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:27.567050 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:27.567238 (+ 188us) server_negotiation.cc:407] Connection header received
0806 06:35:27.567669 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:27.567670 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:27.567679 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:27.567727 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:27.568224 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:27.568681 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:27.569545 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:27.569736 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:27.569781 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:27.571639 (+ 1858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:27.571640 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:27.571640 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:27.571647 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:27.572686 (+ 1039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:27.573008 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:27.573009 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:27.573009 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:27.573050 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:27.573223 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:27.573223 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:27.573223 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:27.573309 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:27.573350 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:27.573540 (+ 190us) server_negotiation.cc:299] Negotiation successful
0806 06:35:27.573593 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 06:35:27.573793 8345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:27.566984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:37427 (local address 127.0.0.1:33600)
0806 06:35:27.567119 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:27.567173 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:27.567597 (+ 424us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:27.567777 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:27.567780 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:27.567987 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:35:27.568164 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:27.568167 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:27.568737 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:27.568737 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:27.569486 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:27.569488 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:27.569819 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:27.569821 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:27.569917 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:27.570317 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:35:27.570326 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:35:27.571550 (+ 1224us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:35:27.572751 (+ 1201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:27.572758 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:27.572759 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:27.572954 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:27.573091 (+ 137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:27.573092 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:27.573092 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:27.573182 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:27.573350 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:35:27.573354 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:35:27.573451 (+ 97us) client_negotiation.cc:769] Sending connection context
0806 06:35:27.573525 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 06:35:27.573628 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":59,"threads_started":1}
I20260806 06:35:27.573954 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 8240
I20260806 06:35:27.574052 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/minicluster-data/master-1/wal/instance
I20260806 06:35:27.574224 7341 external_mini_cluster.cc:1428] Setting key dbe24d11e9345bd36f1c3cf1f026b405
I20260806 06:35:27.574321 8341 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.43.125" port: 44543 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:35:27 dist-test-slave-6zbq krb5kdc[8124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998127, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.125@KRBTEST.COM for kudu/127.7.43.125@KRBTEST.COM
I20260806 06:35:27.581009 8345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:27.574536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.125:44543 (local address 127.0.0.1:58052)
0806 06:35:27.574574 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:27.574584 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:27.574621 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:27.574881 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:27.574882 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:27.574979 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:35:27.575023 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:27.575024 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:27.576039 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:27.576040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:27.576510 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:27.576512 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:27.576919 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:27.576920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:27.576978 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:27.577312 (+ 334us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:35:27.577315 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:35:27.578565 (+ 1250us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:35:27.580050 (+ 1485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:27.580051 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:27.580052 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:27.580250 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:27.580417 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:27.580417 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:27.580418 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:27.580479 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:27.580690 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:35:27.580691 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:35:27.580734 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 06:35:27.580809 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 06:35:27.580896 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260806 06:35:27.581029 8346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:27.574592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58052 (local address 127.7.43.125:44543)
0806 06:35:27.574737 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:27.574741 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:27.574761 (+ 20us) server_negotiation.cc:407] Connection header received
0806 06:35:27.574782 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:27.574784 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:27.574797 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:27.574874 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:27.575080 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:27.575961 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:27.576577 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:27.576858 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:27.576914 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:27.578655 (+ 1741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:27.578662 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:27.578662 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:27.578668 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:27.579972 (+ 1304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:27.580317 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:27.580319 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:27.580319 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:27.580365 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:27.580528 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:27.580529 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:27.580529 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:27.580636 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:27.580689 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:27.580829 (+ 140us) server_negotiation.cc:299] Negotiation successful
0806 06:35:27.580940 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":114,"thread_start_us":82,"threads_started":1}
I20260806 06:35:27.581594 8341 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.43.124" port: 45495 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:35:27.581900 8345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:27.581778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.124:45495 (local address 127.0.0.1:54146)
0806 06:35:27.581830 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:27.581877 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.124:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 06:35:27.582034 8263 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.43.124:45495: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.7.43.124@KRBTEST.COM; defaulting to no policy
I20260806 06:35:27.582453 8345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:27.582327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.124:45495 (local address 127.0.0.1:54156)
0806 06:35:27.582405 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:27.582431 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.124:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260806 06:35:27.582552 8341 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.43.124:45495: Network error: Client connection negotiation failed: client connection to 127.7.43.124:45495: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.43.124@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.43.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.124.keytab.
Entry for principal kudu/127.7.43.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.124.keytab.
I20260806 06:35:27.596573 7341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
/tmp/dist-test-taskfvM9Wt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.124:45495
--webserver_interface=127.7.43.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.124.keytab
--principal=kudu/127.7.43.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.43.84:46317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.43.126:37427,127.7.43.125:44543,127.7.43.124:45495
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:35:27.631774 8341 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.43.124" port: 45495 } attempt: 1
I20260806 06:35:27.632261 8345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:27.632112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.124:45495 (local address 127.0.0.1:54162)
0806 06:35:27.632178 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:27.632229 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.124:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260806 06:35:27.632592 8345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:27.632505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.124:45495 (local address 127.0.0.1:54166)
0806 06:35:27.632547 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:27.632573 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.124:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260806 06:35:27.632697 8341 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.43.124:45495: Network error: Client connection negotiation failed: client connection to 127.7.43.124:45495: connect: Connection refused (error 111)
I20260806 06:35:27.642216 8231 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.43.125" port: 44543 } attempt: 3
Aug 06 06:35:27 dist-test-slave-6zbq krb5kdc[8124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998127, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.126@KRBTEST.COM for kudu/127.7.43.125@KRBTEST.COM
I20260806 06:35:27.649075 8346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:27.642602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58064 (local address 127.7.43.125:44543)
0806 06:35:27.642662 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:27.642664 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:27.642677 (+ 13us) server_negotiation.cc:407] Connection header received
0806 06:35:27.642705 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:27.642706 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:27.642717 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:27.642772 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:27.643053 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:27.643833 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:27.644517 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:27.644849 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:27.644914 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:27.646956 (+ 2042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:27.646958 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:27.646958 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:27.646964 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:27.648120 (+ 1156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:27.648484 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:27.648484 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:27.648485 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:27.648528 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:27.648675 (+ 147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:27.648676 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:27.648676 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:27.648771 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:27.648819 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:27.648961 (+ 142us) server_negotiation.cc:299] Negotiation successful
0806 06:35:27.649018 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 06:35:27.649132 8234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:27.642536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.125:44543 (local address 127.0.0.1:58064)
0806 06:35:27.642593 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:27.642604 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:27.642658 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:27.642791 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:27.642792 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:27.642932 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:35:27.642994 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:27.642996 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:27.643914 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:27.643915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:27.644447 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:27.644449 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:27.644915 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:27.644916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:27.644963 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:27.645409 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:35:27.645413 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:35:27.646883 (+ 1470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:35:27.648195 (+ 1312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:27.648196 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:27.648197 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:27.648420 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:27.648567 (+ 147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:27.648567 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:27.648568 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:27.648629 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:27.648822 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:35:27.648823 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:35:27.648872 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 06:35:27.648945 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 06:35:27.649038 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 06:35:27.649534 8231 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.43.124" port: 45495 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:35:27.649853 8234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:27.649749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.124:45495 (local address 127.0.0.1:54182)
0806 06:35:27.649795 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:27.649828 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.124:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 06:35:27.650143 8234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:27.650064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.124:45495 (local address 127.0.0.1:54196)
0806 06:35:27.650100 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:27.650125 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.124:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260806 06:35:27.650223 8231 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.43.124:45495: Network error: Client connection negotiation failed: client connection to 127.7.43.124:45495: connect: Connection refused (error 111)
W20260806 06:35:27.677572 8350 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 06:35:27.677757 8350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:35:27.677774 8350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:35:27.677791 8350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:35:27.678860 8350 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:35:27.678893 8350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:35:27.678906 8350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:35:27.678917 8350 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:35:27.678927 8350 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:35:27.680284 8350 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.43.84:46317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.43.126:37427,127.7.43.125:44543,127.7.43.124:45495
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/krb5kdc/kudu_127.7.43.124.keytab
--principal=kudu/127.7.43.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.43.124:45495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.43.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8350
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:35:27.680457 8350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:35:27.680614 8350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:35:27.681366 8231 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.43.124" port: 45495 } attempt: 1
I20260806 06:35:27.681700 8234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:27.681590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.124:45495 (local address 127.0.0.1:54200)
0806 06:35:27.681637 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:27.681673 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.124:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 06:35:27.682051 8234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:27.681951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.124:45495 (local address 127.0.0.1:54206)
0806 06:35:27.681995 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:27.682025 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.124:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 06:35:27.682157 8231 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.43.124:45495: Network error: Client connection negotiation failed: client connection to 127.7.43.124:45495: connect: Connection refused (error 111)
W20260806 06:35:27.685189 8356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:27.685202 8358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:27.685286 8350 server_base.cc:1034] running on GCE node
W20260806 06:35:27.685200 8361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:27.685564 8350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:35:27.685827 8350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:35:27.686956 8350 hybrid_clock.cc:648] HybridClock initialized: now 1785998127686946 us; error 26 us; skew 500 ppm
Aug 06 06:35:27 dist-test-slave-6zbq krb5kdc[8124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998127, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:35:27.689078 8350 init.cc:377] Logged in from keytab as kudu/127.7.43.124@KRBTEST.COM (short username kudu)
I20260806 06:35:27.689453 8350 webserver.cc:460] Webserver started at http://127.7.43.124:39891/ using document root <none> and password file <none>
I20260806 06:35:27.689633 8350 fs_manager.cc:359] Metadata directory not provided
I20260806 06:35:27.689678 8350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:35:27.689803 8350 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:35:27.690825 8350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/minicluster-data/master-2/data/instance:
uuid: "dde835b5c4ba4d6d9c751c1331630c7d"
format_stamp: "Formatted at 2026-08-06 06:35:27 on dist-test-slave-6zbq"
server_key: "a27e305e730a9b8a4440d05a9301f0ad"
server_key_iv: "ee8e80807d6d520b2a4163c8704d650e"
server_key_version: "encryptionkey@0"
I20260806 06:35:27.691139 8350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/minicluster-data/master-2/wal/instance:
uuid: "dde835b5c4ba4d6d9c751c1331630c7d"
format_stamp: "Formatted at 2026-08-06 06:35:27 on dist-test-slave-6zbq"
server_key: "a27e305e730a9b8a4440d05a9301f0ad"
server_key_iv: "ee8e80807d6d520b2a4163c8704d650e"
server_key_version: "encryptionkey@0"
I20260806 06:35:27.692292 8350 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:35:27.692893 8370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:27.693041 8350 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:35:27.693120 8350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/minicluster-data/master-2/data,/tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/minicluster-data/master-2/wal
uuid: "dde835b5c4ba4d6d9c751c1331630c7d"
format_stamp: "Formatted at 2026-08-06 06:35:27 on dist-test-slave-6zbq"
server_key: "a27e305e730a9b8a4440d05a9301f0ad"
server_key_iv: "ee8e80807d6d520b2a4163c8704d650e"
server_key_version: "encryptionkey@0"
I20260806 06:35:27.693179 8350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:35:27.700845 8341 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.43.124" port: 45495 } attempt: 2
I20260806 06:35:27.701165 8345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:27.701051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.124:45495 (local address 127.0.0.1:54218)
0806 06:35:27.701107 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:27.701131 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.124:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260806 06:35:27.701457 8345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:27.701366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.124:45495 (local address 127.0.0.1:54224)
0806 06:35:27.701400 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:27.701422 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.43.124:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260806 06:35:27.701531 8341 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.43.124:45495: Network error: Client connection negotiation failed: client connection to 127.7.43.124:45495: connect: Connection refused (error 111)
I20260806 06:35:27.705447 8350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:35:27.707152 8350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:35:27.707275 8350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:35:27.710893 8350 rpc_server.cc:307] RPC server started. Bound to: 127.7.43.124:45495
I20260806 06:35:27.710930 8447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.43.124:45495 every 8 connection(s)
I20260806 06:35:27.711199 8350 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/minicluster-data/master-2/data/info.pb
I20260806 06:35:27.712064 8448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:35:27.713430 8448 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.43.126" port: 37427 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:35:27 dist-test-slave-6zbq krb5kdc[8124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998127, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.124@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:35:27.720626 7341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu as pid 8350
I20260806 06:35:27.720733 7341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvM9Wt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785997861828271-7341-0/minicluster-data/master-2/wal/instance
I20260806 06:35:27.720912 7341 external_mini_cluster.cc:1428] Setting key 88541a745920b1a06e6afa70b92bda87
I20260806 06:35:27.721050 8235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:27.714010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33616 (local address 127.7.43.126:37427)
0806 06:35:27.714058 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:27.714060 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:27.714236 (+ 176us) server_negotiation.cc:407] Connection header received
0806 06:35:27.714709 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:27.714710 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:27.714723 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:27.714774 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:27.715306 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:27.715757 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:27.716692 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:27.716878 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:27.716931 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:27.718791 (+ 1860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:27.718792 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:27.718793 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:27.718800 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:27.719828 (+ 1028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:27.720169 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:27.720170 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:27.720171 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:27.720235 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:27.720442 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:27.720443 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:27.720444 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:27.720548 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:27.720605 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:27.720882 (+ 277us) server_negotiation.cc:299] Negotiation successful
0806 06:35:27.720965 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 06:35:27.721187 8452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:27.713983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.126:37427 (local address 127.0.0.1:33616)
0806 06:35:27.714124 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:27.714156 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:27.714620 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:27.714835 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:27.714838 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:27.715050 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:35:27.715223 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:27.715227 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:27.715833 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:27.715834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:27.716608 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:27.716610 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:27.716968 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:27.716970 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:27.717065 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:27.717467 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:35:27.717475 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:35:27.718697 (+ 1222us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:35:27.719899 (+ 1202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:27.719905 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:27.719908 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:27.720104 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:27.720290 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:27.720291 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:27.720291 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:27.720388 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:27.720603 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:35:27.720609 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:35:27.720748 (+ 139us) client_negotiation.cc:769] Sending connection context
0806 06:35:27.720846 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 06:35:27.720984 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":61,"threads_started":1}
I20260806 06:35:27.721647 8448 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.43.125" port: 44543 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:35:27 dist-test-slave-6zbq krb5kdc[8124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998127, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.124@KRBTEST.COM for kudu/127.7.43.125@KRBTEST.COM
Aug 06 06:35:27 dist-test-slave-6zbq krb5kdc[8124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.43.124@KRBTEST.COM
I20260806 06:35:27.728780 8346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:27.721959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58072 (local address 127.7.43.125:44543)
0806 06:35:27.722010 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:27.722013 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:27.722022 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:35:27.722045 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:27.722046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:27.722056 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:27.722120 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:27.722334 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:27.722808 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:27.723542 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:27.723849 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:27.723918 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:27.726484 (+ 2566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:27.726485 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:27.726486 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:27.726493 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:27.727844 (+ 1351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:27.728157 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:27.728159 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:27.728159 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:27.728228 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:27.728370 (+ 142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:27.728371 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:27.728371 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:27.728494 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:27.728559 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:27.728667 (+ 108us) server_negotiation.cc:299] Negotiation successful
0806 06:35:27.728724 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 06:35:27.728811 8452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:27.721901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.125:44543 (local address 127.0.0.1:58072)
0806 06:35:27.721943 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:27.721953 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:27.721992 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:27.722121 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:27.722121 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:27.722227 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:35:27.722270 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:27.722271 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:27.722879 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:27.722880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:27.723476 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:27.723478 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:27.723912 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:27.723913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:27.723957 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:27.724250 (+ 293us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:35:27.724255 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:35:27.726411 (+ 2156us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:35:27.727924 (+ 1513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:27.727924 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:27.727925 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:27.728095 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:27.728288 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:27.728289 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:27.728289 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:27.728333 (+ 44us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:27.728548 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:35:27.728549 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:35:27.728587 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 06:35:27.728649 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 06:35:27.728726 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260806 06:35:27.729096 8448 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.43.124" port: 45495 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:35:27.729727 8454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:27.721393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54240 (local address 127.7.43.124:45495)
0806 06:35:27.721546 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:27.721550 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:27.721567 (+ 17us) server_negotiation.cc:407] Connection header received
0806 06:35:27.721588 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:27.721589 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:27.721600 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:27.721659 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:27.722394 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:27.723145 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:27.724023 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:27.724422 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:27.724490 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:27.727095 (+ 2605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:27.727104 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:27.727104 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:27.727111 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:27.728517 (+ 1406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:27.728886 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:27.728891 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:27.728892 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:27.728950 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:27.729130 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:27.729131 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:27.729131 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:27.729299 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:27.729434 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:27.729533 (+ 99us) server_negotiation.cc:299] Negotiation successful
0806 06:35:27.729632 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":73,"threads_started":1}
Aug 06 06:35:27 dist-test-slave-6zbq krb5kdc[8124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998127, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.124@KRBTEST.COM for kudu/127.7.43.124@KRBTEST.COM
Aug 06 06:35:27 dist-test-slave-6zbq krb5kdc[8124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.43.126@KRBTEST.COM
I20260806 06:35:27.736071 8452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:27.729300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.124:45495 (local address 127.0.0.1:54254)
0806 06:35:27.729339 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:27.729349 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:27.729383 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:27.729577 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:27.729578 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:27.729714 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:35:27.729741 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:27.729742 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:27.730642 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:27.730643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:27.731176 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:27.731178 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:27.731523 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:27.731524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:27.731573 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:27.731896 (+ 323us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:35:27.731900 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:35:27.733083 (+ 1183us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:35:27.735174 (+ 2091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:27.735175 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:27.735176 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:27.735335 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:27.735520 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:27.735520 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:27.735520 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:27.735568 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:27.735776 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:35:27.735777 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:35:27.735821 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 06:35:27.735873 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 06:35:27.735960 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260806 06:35:27.736166 8455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:27.729346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54254 (local address 127.7.43.124:45495)
0806 06:35:27.729465 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:27.729466 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:27.729473 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:35:27.729491 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:27.729491 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:27.729500 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:27.729562 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:27.729940 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:27.730576 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:27.731267 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:27.731471 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:27.731513 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:27.733153 (+ 1640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:27.733154 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:27.733154 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:27.733160 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:27.735102 (+ 1942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:27.735401 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:27.735402 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:27.735403 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:27.735463 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:27.735622 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:27.735623 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:27.735623 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:27.735720 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:27.735768 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:27.735912 (+ 144us) server_negotiation.cc:299] Negotiation successful
0806 06:35:27.736011 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":97,"thread_start_us":68,"threads_started":1}
I20260806 06:35:27.736713 8235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:27.730415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33624 (local address 127.7.43.126:37427)
0806 06:35:27.730446 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:27.730447 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:27.730455 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:35:27.730503 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:27.730504 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:27.730510 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:27.730551 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:27.730964 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:27.731503 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:27.732107 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:27.732380 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:27.732435 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:27.734235 (+ 1800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:27.734236 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:27.734236 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:27.734243 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:27.735520 (+ 1277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:27.735810 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:27.735811 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:27.735811 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:27.735860 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:27.736038 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:27.736039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:27.736039 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:27.736161 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:27.736320 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:27.736569 (+ 249us) server_negotiation.cc:299] Negotiation successful
0806 06:35:27.736635 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260806 06:35:27.737576 8448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dde835b5c4ba4d6d9c751c1331630c7d: Bootstrap starting.
I20260806 06:35:27.738119 8448 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dde835b5c4ba4d6d9c751c1331630c7d: Neither blocks nor log segments found. Creating new log.
I20260806 06:35:27.738423 8448 log.cc:824] T 00000000000000000000000000000000 P dde835b5c4ba4d6d9c751c1331630c7d: Log is configured to *not* fsync() on all Append() calls
I20260806 06:35:27.739145 8448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dde835b5c4ba4d6d9c751c1331630c7d: No bootstrap required, opened a new log
I20260806 06:35:27.740674 8448 raft_consensus.cc:348] T 00000000000000000000000000000000 P dde835b5c4ba4d6d9c751c1331630c7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69f8b2f4d314472eb4dde3c0d5ac8351" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 37427 } } peers { permanent_uuid: "9cc0b4cc918048c5a7e743771b1b5f97" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 44543 } } peers { permanent_uuid: "dde835b5c4ba4d6d9c751c1331630c7d" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 45495 } }
I20260806 06:35:27.740819 8448 raft_consensus.cc:729] T 00000000000000000000000000000000 P dde835b5c4ba4d6d9c751c1331630c7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dde835b5c4ba4d6d9c751c1331630c7d, State: Initialized, Role: FOLLOWER
I20260806 06:35:27.740906 8448 consensus_queue.cc:260] T 00000000000000000000000000000000 P dde835b5c4ba4d6d9c751c1331630c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69f8b2f4d314472eb4dde3c0d5ac8351" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 37427 } } peers { permanent_uuid: "9cc0b4cc918048c5a7e743771b1b5f97" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 44543 } } peers { permanent_uuid: "dde835b5c4ba4d6d9c751c1331630c7d" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 45495 } }
Aug 06 06:35:27 dist-test-slave-6zbq krb5kdc[8124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.43.125@KRBTEST.COM
I20260806 06:35:27.741156 8461 sys_catalog.cc:455] T 00000000000000000000000000000000 P dde835b5c4ba4d6d9c751c1331630c7d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69f8b2f4d314472eb4dde3c0d5ac8351" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 37427 } } peers { permanent_uuid: "9cc0b4cc918048c5a7e743771b1b5f97" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 44543 } } peers { permanent_uuid: "dde835b5c4ba4d6d9c751c1331630c7d" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 45495 } } }
I20260806 06:35:27.741256 8461 sys_catalog.cc:458] T 00000000000000000000000000000000 P dde835b5c4ba4d6d9c751c1331630c7d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:35:27.741259 8448 sys_catalog.cc:564] T 00000000000000000000000000000000 P dde835b5c4ba4d6d9c751c1331630c7d [sys.catalog]: configured and running, proceeding with master startup.
W20260806 06:35:27.743059 8477 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dde835b5c4ba4d6d9c751c1331630c7d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:35:27.743106 8477 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:35:27.744329 8346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:27.737268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58082 (local address 127.7.43.125:44543)
0806 06:35:27.737297 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:27.737298 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:27.737305 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:35:27.737358 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:27.737359 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:27.737367 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:27.737410 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:27.737838 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:27.738309 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:27.739109 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:27.739355 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:27.739404 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:27.741592 (+ 2188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:27.741593 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:27.741593 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:27.741599 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:27.743000 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:27.743364 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:27.743365 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:27.743365 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:27.743420 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:27.743599 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:27.743600 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:27.743600 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:27.743724 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:27.743857 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:27.744187 (+ 330us) server_negotiation.cc:299] Negotiation successful
0806 06:35:27.744251 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260806 06:35:27.743309 8350 master_runner.cc:186] Error getting master registration for 127.7.43.126:37427: OK, instance_id { permanent_uuid: "69f8b2f4d314472eb4dde3c0d5ac8351" instance_seqno: 1785998127418583 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 06:35:27.745325 8350 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 06:35:27.746632 7341 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:35:27.747642 8201 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "69f8b2f4d314472eb4dde3c0d5ac8351"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33624
I20260806 06:35:27.757323 8231 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.43.124" port: 45495 } attempt: 2
Aug 06 06:35:27 dist-test-slave-6zbq krb5kdc[8124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998127, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.126@KRBTEST.COM for kudu/127.7.43.124@KRBTEST.COM
I20260806 06:35:27.764649 8234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:27.757540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.124:45495 (local address 127.0.0.1:54260)
0806 06:35:27.757586 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:27.757598 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:27.757644 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:27.757760 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:27.757761 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:27.757917 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:35:27.757958 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:27.757960 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:27.758848 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:27.758848 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:27.759631 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:27.759633 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:27.760153 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:27.760154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:27.760211 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:27.760599 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:35:27.760603 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:35:27.762063 (+ 1460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:35:27.763654 (+ 1591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:27.763655 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:27.763656 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:27.763863 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:27.764043 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:27.764044 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:27.764044 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:27.764107 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:27.764375 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:35:27.764375 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:35:27.764417 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 06:35:27.764475 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 06:35:27.764557 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260806 06:35:27.764665 8455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:27.757599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54260 (local address 127.7.43.124:45495)
0806 06:35:27.757662 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:27.757664 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:27.757685 (+ 21us) server_negotiation.cc:407] Connection header received
0806 06:35:27.757702 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:27.757703 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:27.757710 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:27.757756 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:27.758022 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:27.758775 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:27.759735 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:27.760049 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:27.760103 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:27.762163 (+ 2060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:27.762165 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:27.762165 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:27.762172 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:27.763567 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:27.763927 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:27.763928 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:27.763928 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:27.763984 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:27.764159 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:27.764160 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:27.764160 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:27.764310 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:27.764372 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:27.764505 (+ 133us) server_negotiation.cc:299] Negotiation successful
0806 06:35:27.764579 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 06:35:27.766024 8231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69f8b2f4d314472eb4dde3c0d5ac8351: Bootstrap starting.
I20260806 06:35:27.766714 8231 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69f8b2f4d314472eb4dde3c0d5ac8351: Neither blocks nor log segments found. Creating new log.
I20260806 06:35:27.766996 8231 log.cc:824] T 00000000000000000000000000000000 P 69f8b2f4d314472eb4dde3c0d5ac8351: Log is configured to *not* fsync() on all Append() calls
I20260806 06:35:27.767666 8231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69f8b2f4d314472eb4dde3c0d5ac8351: No bootstrap required, opened a new log
I20260806 06:35:27.768915 8231 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69f8b2f4d314472eb4dde3c0d5ac8351 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69f8b2f4d314472eb4dde3c0d5ac8351" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 37427 } } peers { permanent_uuid: "9cc0b4cc918048c5a7e743771b1b5f97" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 44543 } } peers { permanent_uuid: "dde835b5c4ba4d6d9c751c1331630c7d" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 45495 } }
I20260806 06:35:27.769043 8231 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69f8b2f4d314472eb4dde3c0d5ac8351 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69f8b2f4d314472eb4dde3c0d5ac8351, State: Initialized, Role: FOLLOWER
I20260806 06:35:27.769119 8231 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69f8b2f4d314472eb4dde3c0d5ac8351 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69f8b2f4d314472eb4dde3c0d5ac8351" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 37427 } } peers { permanent_uuid: "9cc0b4cc918048c5a7e743771b1b5f97" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 44543 } } peers { permanent_uuid: "dde835b5c4ba4d6d9c751c1331630c7d" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 45495 } }
I20260806 06:35:27.769359 8480 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69f8b2f4d314472eb4dde3c0d5ac8351 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69f8b2f4d314472eb4dde3c0d5ac8351" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 37427 } } peers { permanent_uuid: "9cc0b4cc918048c5a7e743771b1b5f97" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 44543 } } peers { permanent_uuid: "dde835b5c4ba4d6d9c751c1331630c7d" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 45495 } } }
I20260806 06:35:27.769419 8231 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69f8b2f4d314472eb4dde3c0d5ac8351 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:35:27.769436 8480 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69f8b2f4d314472eb4dde3c0d5ac8351 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:35:27.769547 8201 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69f8b2f4d314472eb4dde3c0d5ac8351 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 06:35:27.769627 8201 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69f8b2f4d314472eb4dde3c0d5ac8351 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:35:27.770361 8201 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69f8b2f4d314472eb4dde3c0d5ac8351 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69f8b2f4d314472eb4dde3c0d5ac8351" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 37427 } } peers { permanent_uuid: "9cc0b4cc918048c5a7e743771b1b5f97" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 44543 } } peers { permanent_uuid: "dde835b5c4ba4d6d9c751c1331630c7d" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 45495 } }
I20260806 06:35:27.770865 8201 leader_election.cc:288] T 00000000000000000000000000000000 P 69f8b2f4d314472eb4dde3c0d5ac8351 [CANDIDATE]: Term 1 election: Requested vote from peers 9cc0b4cc918048c5a7e743771b1b5f97 (127.7.43.125:44543), dde835b5c4ba4d6d9c751c1331630c7d (127.7.43.124:45495)
I20260806 06:35:27.771831 8412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "69f8b2f4d314472eb4dde3c0d5ac8351" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "dde835b5c4ba4d6d9c751c1331630c7d"
I20260806 06:35:27.771950 8412 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dde835b5c4ba4d6d9c751c1331630c7d [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:35:27.772773 8412 raft_consensus.cc:2454] T 00000000000000000000000000000000 P dde835b5c4ba4d6d9c751c1331630c7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69f8b2f4d314472eb4dde3c0d5ac8351 in term 1.
I20260806 06:35:27.773068 8163 leader_election.cc:302] T 00000000000000000000000000000000 P 69f8b2f4d314472eb4dde3c0d5ac8351 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69f8b2f4d314472eb4dde3c0d5ac8351, dde835b5c4ba4d6d9c751c1331630c7d; no voters:
I20260806 06:35:27.773182 8480 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69f8b2f4d314472eb4dde3c0d5ac8351 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:35:27.773310 8480 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69f8b2f4d314472eb4dde3c0d5ac8351 [term 1 LEADER]: Becoming Leader. State: Replica: 69f8b2f4d314472eb4dde3c0d5ac8351, State: Running, Role: LEADER
I20260806 06:35:27.773386 8480 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69f8b2f4d314472eb4dde3c0d5ac8351 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69f8b2f4d314472eb4dde3c0d5ac8351" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 37427 } } peers { permanent_uuid: "9cc0b4cc918048c5a7e743771b1b5f97" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 44543 } } peers { permanent_uuid: "dde835b5c4ba4d6d9c751c1331630c7d" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 45495 } }
I20260806 06:35:27.771829 8140 master_runner.cc:186] Error getting master registration for 127.7.43.125:44543: OK, instance_id { permanent_uuid: "9cc0b4cc918048c5a7e743771b1b5f97" instance_seqno: 1785998127562941 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 06:35:27.774610 8140 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 06:35:27.775858 8499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69f8b2f4d314472eb4dde3c0d5ac8351 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69f8b2f4d314472eb4dde3c0d5ac8351. Latest consensus state: current_term: 1 leader_uuid: "69f8b2f4d314472eb4dde3c0d5ac8351" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69f8b2f4d314472eb4dde3c0d5ac8351" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 37427 } } peers { permanent_uuid: "9cc0b4cc918048c5a7e743771b1b5f97" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 44543 } } peers { permanent_uuid: "dde835b5c4ba4d6d9c751c1331630c7d" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 45495 } } }
I20260806 06:35:27.776335 8499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69f8b2f4d314472eb4dde3c0d5ac8351 [sys.catalog]: This master's current role is: LEADER
W20260806 06:35:27.776602 8493 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 69f8b2f4d314472eb4dde3c0d5ac8351: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:35:27.776751 8493 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:35:27.776912 8505 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:35:27.777102 8505 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:35:27.779202 8412 raft_consensus.cc:1261] T 00000000000000000000000000000000 P dde835b5c4ba4d6d9c751c1331630c7d [term 1 FOLLOWER]: Refusing update from remote peer 69f8b2f4d314472eb4dde3c0d5ac8351: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:35:27.779541 8499 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 69f8b2f4d314472eb4dde3c0d5ac8351 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dde835b5c4ba4d6d9c751c1331630c7d" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 45495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:35:27.780841 8461 sys_catalog.cc:455] T 00000000000000000000000000000000 P dde835b5c4ba4d6d9c751c1331630c7d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69f8b2f4d314472eb4dde3c0d5ac8351. Latest consensus state: current_term: 1 leader_uuid: "69f8b2f4d314472eb4dde3c0d5ac8351" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69f8b2f4d314472eb4dde3c0d5ac8351" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 37427 } } peers { permanent_uuid: "9cc0b4cc918048c5a7e743771b1b5f97" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 44543 } } peers { permanent_uuid: "dde835b5c4ba4d6d9c751c1331630c7d" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 45495 } } }
I20260806 06:35:27.780922 8461 sys_catalog.cc:458] T 00000000000000000000000000000000 P dde835b5c4ba4d6d9c751c1331630c7d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:35:27.782102 8499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69f8b2f4d314472eb4dde3c0d5ac8351 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "69f8b2f4d314472eb4dde3c0d5ac8351" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69f8b2f4d314472eb4dde3c0d5ac8351" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 37427 } } peers { permanent_uuid: "9cc0b4cc918048c5a7e743771b1b5f97" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 44543 } } peers { permanent_uuid: "dde835b5c4ba4d6d9c751c1331630c7d" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 45495 } } }
I20260806 06:35:27.782178 8499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69f8b2f4d314472eb4dde3c0d5ac8351 [sys.catalog]: This master's current role is: LEADER
I20260806 06:35:27.782610 8461 sys_catalog.cc:455] T 00000000000000000000000000000000 P dde835b5c4ba4d6d9c751c1331630c7d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "69f8b2f4d314472eb4dde3c0d5ac8351" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69f8b2f4d314472eb4dde3c0d5ac8351" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 37427 } } peers { permanent_uuid: "9cc0b4cc918048c5a7e743771b1b5f97" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 44543 } } peers { permanent_uuid: "dde835b5c4ba4d6d9c751c1331630c7d" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 45495 } } }
I20260806 06:35:27.782688 8461 sys_catalog.cc:458] T 00000000000000000000000000000000 P dde835b5c4ba4d6d9c751c1331630c7d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:35:27.782932 8511 mvcc.cc:205] Tried to move back new op lower bound from 7315448331381223424 to 7315448331360395264. Current Snapshot: MvccSnapshot[applied={T|T < 7315448331381223424}]
I20260806 06:35:27.783458 8513 mvcc.cc:205] Tried to move back new op lower bound from 7315448331381223424 to 7315448331360395264. Current Snapshot: MvccSnapshot[applied={T|T < 7315448331381223424}]
I20260806 06:35:27.783953 8505 catalog_manager.cc:1347] Generated new cluster ID: 194b9a030ac84e1fa11b110b19c20d6f
I20260806 06:35:27.783998 8505 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:35:27.791986 8505 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:35:27.792649 8341 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.43.124" port: 45495 } attempt: 3
I20260806 06:35:27.792996 8505 catalog_manager.cc:1504] Loading token signing keys...
Aug 06 06:35:27 dist-test-slave-6zbq krb5kdc[8124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998127, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.43.125@KRBTEST.COM for kudu/127.7.43.124@KRBTEST.COM
I20260806 06:35:27.801927 8345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:27.792869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.43.124:45495 (local address 127.0.0.1:54280)
0806 06:35:27.792955 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:27.792965 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:27.793001 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:27.793197 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:27.793198 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:27.793357 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:35:27.793392 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:27.793394 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:27.794253 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:27.794254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:27.795520 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:27.795523 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:27.796051 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:27.796052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:27.796117 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:27.796586 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:35:27.796591 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:35:27.798947 (+ 2356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:35:27.800794 (+ 1847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:27.800795 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:27.800795 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:27.801005 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:27.801249 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:27.801250 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:27.801250 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:27.801321 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:27.801611 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:35:27.801612 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:35:27.801667 (+ 55us) client_negotiation.cc:769] Sending connection context
0806 06:35:27.801725 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 06:35:27.801827 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260806 06:35:27.802285 8454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:27.792969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54280 (local address 127.7.43.124:45495)
0806 06:35:27.793017 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:27.793019 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:27.793057 (+ 38us) server_negotiation.cc:407] Connection header received
0806 06:35:27.793076 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:27.793077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:27.793087 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:27.793143 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:27.793471 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:27.794169 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:27.795642 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:27.795951 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:27.795996 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:27.799031 (+ 3035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:27.799032 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:27.799033 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:27.799039 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:27.800681 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:27.801103 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:27.801104 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:27.801104 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:27.801172 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:27.801391 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:27.801392 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:27.801392 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:27.801523 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:27.801574 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:27.802122 (+ 548us) server_negotiation.cc:299] Negotiation successful
0806 06:35:27.802199 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 06:35:27.803012 8235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:27.775157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33632 (local address 127.7.43.126:37427)
0806 06:35:27.775191 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:27.775193 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:27.776309 (+ 1116us) server_negotiation.cc:407] Connection header received
0806 06:35:27.776463 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:27.776464 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:27.776475 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:27.776514 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:27.777649 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:27.778397 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:27.784122 (+ 5725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:27.784411 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:27.784450 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:27.791288 (+ 6838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:27.791290 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:27.791290 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:27.791297 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:27.792794 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:27.794087 (+ 1293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:27.794088 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:27.794088 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:27.794159 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:27.794939 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:27.794940 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:27.794940 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:27.795081 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:27.795154 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:27.802845 (+ 7691us) server_negotiation.cc:299] Negotiation successful
0806 06:35:27.802939 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:35:27.803779 8341 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9cc0b4cc918048c5a7e743771b1b5f97: Bootstrap starting.
I20260806 06:35:27.804535 8341 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9cc0b4cc918048c5a7e743771b1b5f97: Neither blocks nor log segments found. Creating new log.
I20260806 06:35:27.804864 8341 log.cc:824] T 00000000000000000000000000000000 P 9cc0b4cc918048c5a7e743771b1b5f97: Log is configured to *not* fsync() on all Append() calls
I20260806 06:35:27.806398 8346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:27.775357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58098 (local address 127.7.43.125:44543)
0806 06:35:27.775390 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:27.775392 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:27.779078 (+ 3686us) server_negotiation.cc:407] Connection header received
0806 06:35:27.779405 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:27.779406 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:27.779426 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:27.779453 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:27.782301 (+ 2848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:27.783061 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:27.788373 (+ 5312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:27.788689 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:27.788738 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:27.795952 (+ 7214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:27.795953 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:27.795954 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:27.795960 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:27.798632 (+ 2672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:27.804780 (+ 6148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:27.804781 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:27.804782 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:27.804846 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:27.805547 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:27.805560 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:27.805560 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:27.805682 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:27.805762 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:27.806187 (+ 425us) server_negotiation.cc:299] Negotiation successful
0806 06:35:27.806273 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 06:35:27.806754 8455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:27.775972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54266 (local address 127.7.43.124:45495)
0806 06:35:27.776002 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:27.776004 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:27.779013 (+ 3009us) server_negotiation.cc:407] Connection header received
0806 06:35:27.779519 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:27.779520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:27.779541 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:27.779567 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:27.783296 (+ 3729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:27.783999 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:27.788824 (+ 4825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:27.789129 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:27.789166 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:27.796348 (+ 7182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:27.796349 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:27.796349 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:27.796356 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:27.798077 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:27.804475 (+ 6398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:27.804489 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:27.804489 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:27.804544 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:27.805260 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:27.805261 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:27.805262 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:27.805394 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:27.805481 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:27.806592 (+ 1111us) server_negotiation.cc:299] Negotiation successful
0806 06:35:27.806662 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260806 06:35:27.808544 8505 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 69f8b2f4d314472eb4dde3c0d5ac8351: Generated new TSK 0
I20260806 06:35:27.808738 8505 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:35:27.814944 8341 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9cc0b4cc918048c5a7e743771b1b5f97: No bootstrap required, opened a new log
I20260806 06:35:27.816627 8341 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9cc0b4cc918048c5a7e743771b1b5f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69f8b2f4d314472eb4dde3c0d5ac8351" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 37427 } } peers { permanent_uuid: "9cc0b4cc918048c5a7e743771b1b5f97" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 44543 } } peers { permanent_uuid: "dde835b5c4ba4d6d9c751c1331630c7d" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 45495 } }
I20260806 06:35:27.816732 8341 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9cc0b4cc918048c5a7e743771b1b5f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cc0b4cc918048c5a7e743771b1b5f97, State: Initialized, Role: FOLLOWER
I20260806 06:35:27.816792 8341 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9cc0b4cc918048c5a7e743771b1b5f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69f8b2f4d314472eb4dde3c0d5ac8351" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 37427 } } peers { permanent_uuid: "9cc0b4cc918048c5a7e743771b1b5f97" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 44543 } } peers { permanent_uuid: "dde835b5c4ba4d6d9c751c1331630c7d" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 45495 } }
I20260806 06:35:27.817005 8521 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9cc0b4cc918048c5a7e743771b1b5f97 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69f8b2f4d314472eb4dde3c0d5ac8351" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 37427 } } peers { permanent_uuid: "9cc0b4cc918048c5a7e743771b1b5f97" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 44543 } } peers { permanent_uuid: "dde835b5c4ba4d6d9c751c1331630c7d" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 45495 } } }
I20260806 06:35:27.817054 8341 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9cc0b4cc918048c5a7e743771b1b5f97 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:35:27.817224 8304 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9cc0b4cc918048c5a7e743771b1b5f97 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:35:27.818125 8304 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9cc0b4cc918048c5a7e743771b1b5f97 [term 1 FOLLOWER]: Refusing update from remote peer 69f8b2f4d314472eb4dde3c0d5ac8351: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:35:27.818459 8480 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 69f8b2f4d314472eb4dde3c0d5ac8351 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cc0b4cc918048c5a7e743771b1b5f97" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 44543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:35:27.818171 8521 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9cc0b4cc918048c5a7e743771b1b5f97 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 06:35:27.818924 8538 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9cc0b4cc918048c5a7e743771b1b5f97: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:35:27.818966 8538 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:35:27.819221 8305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "69f8b2f4d314472eb4dde3c0d5ac8351" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "9cc0b4cc918048c5a7e743771b1b5f97"
I20260806 06:35:27.819957 8305 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9cc0b4cc918048c5a7e743771b1b5f97 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69f8b2f4d314472eb4dde3c0d5ac8351 in term 1.
I20260806 06:35:27.823237 8521 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9cc0b4cc918048c5a7e743771b1b5f97 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69f8b2f4d314472eb4dde3c0d5ac8351. Latest consensus state: current_term: 1 leader_uuid: "69f8b2f4d314472eb4dde3c0d5ac8351" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69f8b2f4d314472eb4dde3c0d5ac8351" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 37427 } } peers { permanent_uuid: "9cc0b4cc918048c5a7e743771b1b5f97" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 44543 } } peers { permanent_uuid: "dde835b5c4ba4d6d9c751c1331630c7d" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 45495 } } }
I20260806 06:35:27.823334 8521 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9cc0b4cc918048c5a7e743771b1b5f97 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:35:27.823374 8521 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9cc0b4cc918048c5a7e743771b1b5f97 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "69f8b2f4d314472eb4dde3c0d5ac8351" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69f8b2f4d314472eb4dde3c0d5ac8351" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 37427 } } peers { permanent_uuid: "9cc0b4cc918048c5a7e743771b1b5f97" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 44543 } } peers { permanent_uuid: "dde835b5c4ba4d6d9c751c1331630c7d" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 45495 } } }
I20260806 06:35:27.823427 8521 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9cc0b4cc918048c5a7e743771b1b5f97 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:35:27.823879 8541 mvcc.cc:205] Tried to move back new op lower bound from 7315448331497787392 to 7315448331360395264. Current Snapshot: MvccSnapshot[applied={T|T < 7315448331381223424}]
W20260806 06:35:27.832078 8235 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:35:27.830636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33648 (local address 127.7.43.126:37427)
0806 06:35:27.830674 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:27.830675 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:27.831475 (+ 800us) server_negotiation.cc:407] Connection header received
0806 06:35:27.831570 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:27.831571 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:27.831581 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:27.831609 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:27.832023 (+ 414us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33648: BlockingRecv error: recv got EOF from 127.0.0.1:33648 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260806 06:35:27.832852 8455 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:35:27.830959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54296 (local address 127.7.43.124:45495)
0806 06:35:27.831004 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:27.831006 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:27.832385 (+ 1379us) server_negotiation.cc:407] Connection header received
0806 06:35:27.832473 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:27.832474 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:27.832483 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:27.832509 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:27.832796 (+ 287us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54296: BlockingRecv error: recv got EOF from 127.0.0.1:54296 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260806 06:35:27.833580 8346 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:35:27.831221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58104 (local address 127.7.43.125:44543)
0806 06:35:27.831254 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:27.831256 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:27.833143 (+ 1887us) server_negotiation.cc:407] Connection header received
0806 06:35:27.833225 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:27.833226 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:27.833235 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:27.833262 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:27.833529 (+ 267us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58104: BlockingRecv error: recv got EOF from 127.0.0.1:58104 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260806 06:35:27.835364 8346 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:35:27.834713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58114 (local address 127.7.43.125:44543)
0806 06:35:27.834743 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:27.834745 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:27.834914 (+ 169us) server_negotiation.cc:407] Connection header received
0806 06:35:27.835007 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:27.835008 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:27.835024 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:27.835050 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:27.835324 (+ 274us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58114: BlockingRecv error: recv got EOF from 127.0.0.1:58114 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260806 06:35:27.838217 8346 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:35:27.837549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58124 (local address 127.7.43.125:44543)
0806 06:35:27.837580 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:27.837581 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:27.837767 (+ 186us) server_negotiation.cc:407] Connection header received
0806 06:35:27.837868 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:27.837868 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:27.837877 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:27.837903 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:27.838176 (+ 273us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58124: BlockingRecv error: recv got EOF from 127.0.0.1:58124 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260806 06:35:27.842054 8346 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:35:27.841374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58132 (local address 127.7.43.125:44543)
0806 06:35:27.841411 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:27.841413 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:27.841590 (+ 177us) server_negotiation.cc:407] Connection header received
0806 06:35:27.841696 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:27.841697 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:27.841706 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:27.841731 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:27.842017 (+ 286us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58132: BlockingRecv error: recv got EOF from 127.0.0.1:58132 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260806 06:35:27.847898 8346 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:35:27.847230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58140 (local address 127.7.43.125:44543)
0806 06:35:27.847259 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:27.847260 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:27.847445 (+ 185us) server_negotiation.cc:407] Connection header received
0806 06:35:27.847529 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:27.847530 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:27.847540 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:27.847567 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:27.847858 (+ 291us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58140: BlockingRecv error: recv got EOF from 127.0.0.1:58140 (error 108)
Metrics: {"server-negotiator.queue_time_us":14}
W20260806 06:35:27.857607 8346 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:35:27.856966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58150 (local address 127.7.43.125:44543)
0806 06:35:27.856996 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:27.856998 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:27.857179 (+ 181us) server_negotiation.cc:407] Connection header received
0806 06:35:27.857262 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:27.857263 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:27.857272 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:27.857298 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:27.857567 (+ 269us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58150: BlockingRecv error: recv got EOF from 127.0.0.1:58150 (error 108)
Metrics: {"server-negotiator.queue_time_us":14}
W20260806 06:35:27.875645 8346 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:35:27.874841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58158 (local address 127.7.43.125:44543)
0806 06:35:27.874879 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:27.874881 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:27.875109 (+ 228us) server_negotiation.cc:407] Connection header received
0806 06:35:27.875210 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:27.875211 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:27.875222 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:27.875251 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:27.875599 (+ 348us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58158: BlockingRecv error: recv got EOF from 127.0.0.1:58158 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260806 06:35:27.910121 8346 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:35:27.909142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58170 (local address 127.7.43.125:44543)
0806 06:35:27.909188 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:27.909190 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:27.909454 (+ 264us) server_negotiation.cc:407] Connection header received
0806 06:35:27.909562 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:27.909563 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:27.909576 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:27.909605 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:27.910065 (+ 460us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58170: BlockingRecv error: recv got EOF from 127.0.0.1:58170 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260806 06:35:27.976364 8346 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:35:27.975732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58172 (local address 127.7.43.125:44543)
0806 06:35:27.975793 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:27.975795 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:27.975898 (+ 103us) server_negotiation.cc:407] Connection header received
0806 06:35:27.975964 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:27.975965 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:27.975976 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:27.976015 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:27.976303 (+ 288us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58172: BlockingRecv error: recv got EOF from 127.0.0.1:58172 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260806 06:35:28.106106 8346 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:35:28.105393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58180 (local address 127.7.43.125:44543)
0806 06:35:28.105457 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:28.105459 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:28.105527 (+ 68us) server_negotiation.cc:407] Connection header received
0806 06:35:28.105578 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:28.105578 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:28.105589 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:28.105638 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:28.106026 (+ 388us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58180: BlockingRecv error: recv got EOF from 127.0.0.1:58180 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260806 06:35:28.363770 8346 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:35:28.363146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58186 (local address 127.7.43.125:44543)
0806 06:35:28.363207 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:28.363209 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:28.363264 (+ 55us) server_negotiation.cc:407] Connection header received
0806 06:35:28.363337 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:28.363338 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:28.363349 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:28.363393 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:28.363723 (+ 330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58186: BlockingRecv error: recv got EOF from 127.0.0.1:58186 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 06:35:28.431023 8631 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69f8b2f4d314472eb4dde3c0d5ac8351 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "69f8b2f4d314472eb4dde3c0d5ac8351" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69f8b2f4d314472eb4dde3c0d5ac8351" member_type: VOTER last_known_addr { host: "127.7.43.126" port: 37427 } } peers { permanent_uuid: "9cc0b4cc918048c5a7e743771b1b5f97" member_type: VOTER last_known_addr { host: "127.7.43.125" port: 44543 } } peers { permanent_uuid: "dde835b5c4ba4d6d9c751c1331630c7d" member_type: VOTER last_known_addr { host: "127.7.43.124" port: 45495 } } }
I20260806 06:35:28.431134 8631 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69f8b2f4d314472eb4dde3c0d5ac8351 [sys.catalog]: This master's current role is: LEADER
I20260806 06:35:28.743492 8477 catalog_manager.cc:1259] Loaded cluster ID: 194b9a030ac84e1fa11b110b19c20d6f
I20260806 06:35:28.743553 8477 catalog_manager.cc:1552] T 00000000000000000000000000000000 P dde835b5c4ba4d6d9c751c1331630c7d: loading cluster ID for follower catalog manager: success
I20260806 06:35:28.744298 8477 catalog_manager.cc:1574] T 00000000000000000000000000000000 P dde835b5c4ba4d6d9c751c1331630c7d: acquiring CA information for follower catalog manager: success
I20260806 06:35:28.744541 8477 catalog_manager.cc:1602] T 00000000000000000000000000000000 P dde835b5c4ba4d6d9c751c1331630c7d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 06:35:28.819346 8538 catalog_manager.cc:1259] Loaded cluster ID: 194b9a030ac84e1fa11b110b19c20d6f
I20260806 06:35:28.819410 8538 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9cc0b4cc918048c5a7e743771b1b5f97: loading cluster ID for follower catalog manager: success
I20260806 06:35:28.820505 8538 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9cc0b4cc918048c5a7e743771b1b5f97: acquiring CA information for follower catalog manager: success
I20260806 06:35:28.820815 8538 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9cc0b4cc918048c5a7e743771b1b5f97: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 06:35:28.879323 8644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:28.876812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58192 (local address 127.7.43.125:44543)
0806 06:35:28.876933 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:28.876935 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:28.876960 (+ 25us) server_negotiation.cc:407] Connection header received
0806 06:35:28.877043 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:28.877044 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:28.877097 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:28.877165 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:35:28.877318 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:28.877815 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:28.878577 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:28.878783 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:28.878838 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:28.878997 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:35:28.879104 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:35:28.879170 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:28.879219 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 06:35:28.879241 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":55,"threads_started":1}
I20260806 06:35:28.883028 8656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:28.880914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54304 (local address 127.7.43.124:45495)
0806 06:35:28.881004 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:28.881006 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:28.881014 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:35:28.881048 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:28.881049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:28.881081 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:28.881155 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:35:28.881251 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:28.881716 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:28.882405 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:28.882590 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:28.882635 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:28.882775 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:35:28.882875 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:35:28.882932 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:28.882952 (+ 20us) server_negotiation.cc:299] Negotiation successful
0806 06:35:28.882969 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74,"thread_start_us":47,"threads_started":1}
I20260806 06:35:28.886837 8644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:28.884376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58198 (local address 127.7.43.125:44543)
0806 06:35:28.884532 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:28.884534 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:28.884540 (+ 6us) server_negotiation.cc:407] Connection header received
0806 06:35:28.884561 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:28.884561 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:28.884570 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:28.884611 (+ 41us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:35:28.884719 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:28.885364 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:28.886208 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:28.886461 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:28.886507 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:28.886626 (+ 119us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:35:28.886656 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:35:28.886699 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:28.886756 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 06:35:28.886774 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138}
I20260806 06:35:28.887673 8663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:28.884434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33652 (local address 127.7.43.126:37427)
0806 06:35:28.884544 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:28.884546 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:28.884813 (+ 267us) server_negotiation.cc:407] Connection header received
0806 06:35:28.884879 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:28.884880 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:28.884927 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:28.885003 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:35:28.885142 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:28.885866 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:28.886844 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:28.887147 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:28.887203 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:28.887372 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:35:28.887466 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:35:28.887517 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:28.887571 (+ 54us) server_negotiation.cc:299] Negotiation successful
0806 06:35:28.887595 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":61,"threads_started":1}
I20260806 06:35:28.887694 8656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:28.884910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54308 (local address 127.7.43.124:45495)
0806 06:35:28.884945 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:28.884946 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:28.885077 (+ 131us) server_negotiation.cc:407] Connection header received
0806 06:35:28.885130 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:28.885131 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:28.885141 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:28.885187 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:35:28.885295 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:28.885761 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:28.886838 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:28.887121 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:28.887156 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:28.887472 (+ 316us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:35:28.887498 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:35:28.887550 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:28.887606 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 06:35:28.887626 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:35:28.889120 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 8140
I20260806 06:35:28.900729 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 8240
I20260806 06:35:28.907980 7341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvM9Wt/build/release/bin/kudu with pid 8350
2026-08-06T06:35:28Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2820 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5662 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (267076 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST